I don't know what your experiences are with IVE patching but mine have been relatively painless. There are a few easy rules that I follow to ensure that things go well.
First is to know when patching is required. Go to the Juniper support website, login and make sure you are setup for alerts/bulletins. Remember you need to enable yourself for both software and hardware.
Next is test (if you can). If you have a test IVE or a smaller IVE used for a smaller office, upgrade that and test before your main sites. If you have different client builds accessing your IVEs, see if you can get as many of them as possible to connect to the upgraded test box. Have someone (maybe you) create a risks and issues log of all the events during the testing and during the upgrade planning.
Read read READ those release notes. Seriously, read them cover to cover and envisage how the changes apply to your environment.
Prep you clients.
- Don't use too many browsers. I'm aware that the IVE supports a wide range a browsers but for the upgrade it would be easier to limit the browsers that you ask people to use. That way you can easily brief users on all the options/menus/changes etc. IE is the default that I use for Windows clients.
- Make sure they have your IVE URLs in the Trusted Sites of your browsers. This will limit the problem of users missing the security warnings or the browsers potentially blocking the SSLVPN clients when they connect for the first time after the upgrade.
- Use the Juniper Installer Service. In a Windows environments these are a godsend. Two clients are available in your IVE's Maintenance | System | Installer section of the GUI and they come in exe or msi formats. What they do is once they are installed with admin privileges they enable further IVE client installs to occur without the end user needing admin rights. Have a read of the Appendix of the Juniper Client Side Changes (this doc is for v7.1 of the IVE OS) document to see how this helps.
During the upgrade
- Don't panic. JTAC is there for you, contact them here and these upgrades are (touch wood) relatively bulletproof.
- First take backups. Multiple ones, both single file backups and XML backups.
- Clear down the logs before you upgrade. (To speed things up)
- Don't worry about the time taken, I've found that SA's can take a good 20-40 mins to upgrade sometimes. The rule of thumb is when you are about ready to go get your console cable out to check what's going on, that's the time that it will finish upgrading.
- If in a cluster the other IVE's will upgrade after the first one has (and generally they will be quicker than the first one)
- Understand the Rollback Button in Maintenance | System. Any changes you make post upgrade will be lost if you use the Rollback Button. The IVE takes a point in time image to create the rollback image. Any config changes are only written into the live flash memory.
- Have a testing script that you've gone through with your management team and if applicable with user input as to what needs to be tested. That way if your script is completed correctly you've not missed anything important. This is especially useful if your upgrade is taking part in the dead of night, so you do not have to work from a fallible mental list.
Troubleshooting post upgrade
You've upgraded the IVE and now there will be the odd client having connection problems. Generally they will be related to the IVE software loaded on the client and the browser. Remember most of the IVE traffic goes through the browser so any problems with the browser might cause problems with the IVE. Your 3 steps to heaven are:-
- Clear down browser cache. Seriously, this works for 90+% of all post upgrade problems. If you have full control of the clients you could even script this or create a GPO for it ahead of time.
- Get the user to remove/uninstall all the Juniper Software from the client. If you are using the Installer Service they will auto re-install (you are using the service aren't you?) & try again. This fixes most of the remaining problems.
- If you are having problems with Pulse and/or Network Connect on a Windows client then it's time to get technical. This is the final step before going to JTAC and does involve some technical know how & maybe admin rights on the client.
Uninstall pulse, any other vpn client, and anything else loaded into the driver stack. (sniffer wireshark etc… leave AV/Firewall driver for now, but if it still doesn't work try uninstalling it)
Then run from an admin command prompt
netsh int ip reset reset.log
netsh winsock reset
(Only one will work, they are the same command for the various versions of Windows) It will reset and rebuild your IP/TCP stack on the client
Reboot and reinstall.
So far the plan above has kept my upgrades relatively trouble free.
If you have found it useful please leave a comment. Also if you have any tricks or tips of your own please shout out and I'll add them on.
Chris