Because of this, you may unbox your server and find that not everything works “right out of the box”. In the case of our server project, we bought a DL380 Gen 10 that was going to be used as a replication target for Virtual Machine backups. We want to use all three “boxes” of hard drive slots. Pretty simple: just a server full of large form factor SAS drives.
The server came with cables from the SAS ports on the motherboard to all three “backplanes” of the drive boxes, but only the first two boxes recognized that they had drives in them.
Turns out that first of all we had a bad SAS cable on box three, but that wasn’t the only problem. According to HPE support the third box doesn’t work unless you use a separate cable that wasn’t included.
This does make sense considering all of the possible configurations for the server. Presumable the third SAS cable that was installed would work for another common layout. Not ours, though. We needed another cable. Fortunately new servers come with next day support so we were able to get another cable the next day. (Well, the “next day” after two days of arguing with HPE support.)
If you find yourself in this situation, you need the black cable in this picture:
Lately, I’ve been installing enough new servers (or inheriting newly installed servers) that now seems like a good time to show what it’s like to start from scratch with a brand new server. Obviously, no one will have exactly the same implementation as this, but the idea is to show the basics of what it’s like to set up a new server.
This will not cover everything you can do with a server. It’s basically just a way for me to organize my posts into something fun.
This page right here is going to host links that, if followed chronologically, will show what it looks like to build a brand new server straight out of the box. Basically all I’m doing is reordering some of my existing posts, but this page will corral them into a mighty Voltron of a larger project. I will probably go back and edit the old posts to make more sense in the context of this project (or not depending on how much time I have). Don’t worry, the original posts will work as standalones.
Anyway, here’s what it’s like to buy and implement a server!
(This is a work in progress. All of the steps may not be here yet. I’ll remove this disclaimer when I feel it is more complete)
In the previous post I showed how to use PowerShell to find and install Windows Updates. In this one, we will run our PowerShell script every Sunday afternoon using Task Scheduler. The script will install any available Windows Updates, reboot the computer if necessary, and email us the details of the installation.
The Scheduled Task
Open Task Scheduler. One way to do this is to hold the Windows key and press R. In the Run box type: taskschd.msc. In Task Scheduler, right-click on the Task Scheduler Library and choose “Create Basic Task”.
The “Create Basic Task Wizard” will open. Name your task something descriptive.
In my case I want the task to run on Sundays so I choose “Weekly”. Yours may differ.
Your next screen could look different. In my case I chose Weekly so I choose the day of the week here.
Next we choose “Start a Program” because we want to start PowerShell to run our Windows Update script. We will be sending an email also, but that part is in our PowerShell script so we can ignore the “Send and e-mail” option here. It doesn’t work anymore anyway.
For the “Program/Script” field you will either paste in or browse to the .exe for PowerShell. Chances are good it’s the same as mine below. In the “Add Arguments” field you put the path to your .ps1 script file including the file name. For example if it’s on the same computer you’re running the Task Scheduler on it may be C:\Scripts\WUpdate.ps1 or if the script is on a file share it may look like \\Server1\Share\Scripts\WUpdate.ps1. “Start In” can stay blank unless you really need it.
The last page lets you review the task before you create it. If everything looks ok, click “Finish”.
Double-click on the new task. The only thing you should have left to do in the settings is click the radio button next to “Run whether user is logged on or not.” This is important because we will be at the beach on Sunday when the task needs to be run so we will definitely not be logged in. You may also want to change the user account used to run the task, but that depends on your circumstances. Click OK. You will be asked to give the password for the account that will run the task since it will not be logged in at the time the task will run.
Now, to test it! Right-click on the new task and choose “Run”. If everything was done correctly you will get an email that looks like this:
Moving the Scheduled Task to Other Servers
You can export the task to XML so you can import it into another server. All you need to do is right-click the task in Task Scheduler and choose “Export”. Here is my XML. Yours may differ.
To import the task into the second server open Task Scheduler on that server and right-click on “Task Scheduler Library”. This time choose “Import Task” and browse to your XML file that you exported. The settings of the task will automatically open so you can review or change them.
Another annoying thing about Windows Updates is that they often require a reboot and a long time watching percentages slowly rise (Often we find ourselves staring at 100% complete for a half an hour). This is especially bad when dealing with production servers that can only be rebooted on a Sunday.
Windows does have a built-in scheduling for installing Windows Updates, but it often just doesn’t work and you don’t have much flexibility about the timing of when things end up happening. In the newest version of Windows 10 you can delay updates for up to a week, but that’s it. Working hours can be set so the thing doesn’t reboot during the day, but there are limits to that, too.
Another, more fun way to handle Windows Updates is, of course, PowerShell! In this article we are going to learn how to use PowerShell to install Windows Updates and in the next article we will use Task Scheduler to run the script on Sunday afternoon when no one is at work.
(See my previous post for instructions on setting Windows Updates to manual only with Sconfig. Keep in mind Sconfig is only for Windows Server, not Windows 10.)
Unzip the file to C:\Windows\System32\WindowsPowerShell\v1.0\Modules\ and then open PowerShell as an administrator (Right-click to find that option). I like to use PowerShell ISE because it’s got the nice “script pane” built in so you can copy and paste my examples in there and modify them to suit your environment.
To make sure it installed correctly and to see what new functions we have run:
Get-Command –module PSWindowsUpdate
If you get an error like: ” FullyQualifiedErrorId : CommandNotFoundException Get-WUInstall : The term ‘Get-WUInstall’ is not recognized as the name of a cmdlet, function, script file, or operable program” then run this:
This next command will list all of the Windows Updates you can install on your computer, but does not install them.
You may know that in the Settings GUI version of Windows Update you can use Windows Update to get updates for other things besides Windows, like Microsoft Office. You can do that with PowerShell, too! It just requires one extra step in the initial setup.
The command for the wider search of Microsoft (as opposed to Windows) Updates is:
Get-WUInstall –MicrosoftUpdate –ListOnly
If you run that one now you will get an error:
Luckily, this is easy to fix. Just run this to register the proper service:
Now run Get-WUInstall –MicrosoftUpdate –ListOnly again:
As you can see, I now have an update for Silverlight in addition to the Windows Defender updates.
Installing The Updates With PowerShell
Now that you’ve seen what updates are ready to install you can install them simply by running:
You will get a popup asking for permission to install the update:
You can click “Yes” or “No” for each indvidual update or click “Yes to All” to install all of them from the first popup. Another approach would be to run this so you don’t have to approve them and they will just install:
Get-WUInstall –MicrosoftUpdate –AcceptAll
You can even add -AutoReboot at the end to have the server automatically reboot after installing the updates if required to do so. This is what we will be doing in our scheduled task so we don’t have to remote in to the server on a Sunday when we’d rather be at the beach:
This what it will look like as it does its thing. Notice the message displayed above the code window as it goes through the steps of Accepted, Downloaded, and Installed:
And when it’s finished installing it will look like this (assuming everything installed correctly):
There is a chance that an update could fail. If one does you will get output like:
4 Failed 63 KB Microsoft driver update for MS Publisher
If you have an update that repeatedly fails to install you could download it from the Microsoft Update Catalog (https://www.catalog.update.microsoft.com/Home.aspx ) and install it manually or you could disable installing the one specific update if you decide you don’t need it.
Disabling Specific Windows Updates
One of the other useful commands is Get-WUHistory. This will show you all of the Windows Updates that installed on the computer, their KB, and the date and time of installation. This is handy for finding updates that broke your computer and need to be uninstalled. You can always Google the KB to find out exactly what the update entails.
If you find one that you want to uninstall, here’s how to do it (replace the KB with the one you want to uninstall):
Get-WUUninstall -KBArticleID KB4501375
To keep Windows from installing that update again you will have to “hide” it or it will install it again later:
Anything beginning with a dollar sign ($) is a variable in PowerShell. The ones above are our email settings. PowerShell will need to know how to send the email so you will need to swap out your values for the parts in bold.
$file = "PathToTranscriptFile"
This variable stores the path to the transcript file to simplify adding it into the body of the email.
Start-Transcript -path $file
Start-Transcript does pretty much what you’d think: it
saves what happens in the PowerShell window to a file. This is how we’re going
to harvest the status of the Windows Updates so we know what got installed.
This is the command we learned in the last post. It
installs all of the available Windows Updates and reboots the computer if
#Get-WUInstall –MicrosoftUpdate –ListOnly
This line with the “#” in front of it will not run. The “#” tells PowerShell to ignore the line and go to the next line. I have this line in here for testing purposes. When I only want to get an email about the available updates, but I don’t want to install them (and I DEFINITELY don’t want to reboot the server) I will move the “#” up one line to expose this test line and hide the one that does the real update. You might want this line for initial testing just in case the email doesn’t work how you need it to at first. When you get everything how you like it you can move the “#” back down a line.
Then we stop the transcript so we can get the file the
transcript created and put it in the body of an email.
$server = $env:COMPUTERNAME
In this line we are creating a variable that holds the name of the server we are running the Windows Updates on. The server name is only useful if you are running this script on multiple computers, which I am.
$env is basically a list of variables that you can use to get (or set) information about your computer. You can run Get-ChildItem Env: to see a list of all of the Environment variables. COMPUTERNAME is the name of the computer, of course, and using the Environment Variable for it enables us to use this same script on any computer; we don’t need to supply the computer’s name because the script gets it for us at runtime!
$body = [IO.File]::ReadAllText($file)
The $body variable reads the $file variable that has our Update info and stores it to use in our email alert.
Finally, we send the email alert. Send-mailmessage is the commandlet that sends an email. It requires a sending email address, a destination email address, a subject line, an email server and the body of the email which contains the details of our Windows Updates.
You can test this script by opening PowerShell ISE as an administrator and loading the script from “File” > “Open” or the folder icon. Don’t forget to move the “#” up to the -AutoReboot line if you don’t actually want to install and reboot the server. Press the green “play” icon to run the script.
If everything was done correctly you should get an email that looks like this:
In the next post I will show you how to set up a scheduled task that will run Windows Updates every Sunday afternoon.
The new versions of Microsoft Windows 10 and and Windows Server have made managing Windows Updates right out of the box kind of a mess. It is not fun for an important server to suddenly reboot itself causing any number of important features to be interrupted. The newest version of Windows 10 as of this writing has only just allowed us to postpone Windows updates and even then they only allow a delay of a week. Windows Server will force a reboot if left with the default settings.
There are a number of ways to fix this such as using Windows
Server Update Services (WSUS) or a Long-Term Servicing Channel (LTSC) version
of Windows. You can also use Group Policy to control some aspects of Windows
In this example we will stop Windows Updates from installing
using Sconfig and then the next article will be about scripting the updates
Sconfig is built into Windows Server so you don’t even have
to go and download it. To start Sconfig run a Command Prompt as an
administrator. When it opens simply type: “sconfig” without the quotes and hit
A new, blue-colored Command Prompt will open:
You can try out the other options, but the one we’re looking
for is number 5: “Windows Update Settings”. Set it to “M” for Manual.
You will know it worked when you see this:
Double-check the Sconfig menu to make sure it says Manual
This will ensure Windows Server will not try to install any
updates unless you tell it to. This can be set in Group Policy also, but in my experience
it doesn’t always work and a server may still try to update and reboot unless
you use Sconfig and set Windows Updates to manual. It’s worth taking a few
minutes just to be sure.
Keep in mind that Sconfig is only for Windows Server. It
doesn’t work with Windows 10.
For some of you just setting updates to manual with Sconfig will
be enough to relieve the stress of unexpected server reboots. If you want to
schedule the updates to happen, for example, every Sunday afternoon when no one
is working, the Powershell fun is coming up in the next article.
One of the most important tools you have as a system administrator is the simple email alert. No matter what the problem is, it’s great to have a head start on fixing it before the phone starts ringing. In the case of HP servers you can get an email alert so you can replace that failed part in your server before it brings the whole thing down. It just needs to be set up first, which is not super simple if you haven’t done it before. Luckily for you I have this nice little tutorial to walk you through it.
The first thing I should mention is that there are little differences in the way certain generations of servers (and different versions of the software) handle this process. I have used the methods described below on Gen 6 through Gen 9 servers. For the most part they are pretty similar, but you may see something on your server that doesn’t look quite like what I have here. Keeping that in mind I have tried to point out any discrepancies I have seen and to give you as much information as I can so you can hopefully bridge any gaps you may come across.
Also, you may notice the names Hewlett-Packard, HP and HPE used interchangeably. In fact, HPE (Hewlett Packard Enterprise) is now a separate company from HP (Hewlett-Packard). They split around 2016 and now HP sells computers and printers and HPE handles servers. Doesn’t really matter to us except that HPE is starting to change things and that’s okay because otherwise life would be boring, right? Anyway, let’s get into it.
HPE’s servers use “Insight Management Agents” to monitor the health of their hardware. These are drivers that you need to install on the server. With the agents installed you can set up email alerts to tell you when something goes wrong with the server. On Gen 9 and older servers these are all free downloads. On servers newer than Gen 9 this functionality has been moved into the iLO and requires a paid license. I will cover that in a separate tutorial.
The first thing we are going to install is the “iLO Management Controller Driver Package”. The version you want will depend on the server’s iLO version. For example, the download for iLO 3 and 4 for Windows Server 2016 and 2019 which I will be using in the examples below is here: https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_e20968c891b444c6b6de68a734
If you need a different one, you can probably find it here: https://support.hpe.com/hpesc/public/km/search#q=Management%20Controller%20Driver%20Package&t=All&sort=relevancy
Once the “iLO Management Controller Driver Package” downloads, right-click it and choose “Run as administrator”. You will see a setup wizard that looks like this:
Click the “Install” button and let the installer run its course.
Next, download and install the “Channel Interface Driver” for your operating system. You should get at least version 188.8.131.52. If you don’t there is a very real chance of random crashes. I have seen it myself. There is a bug in the older versions of the driver (https://downloads.hpe.com/pub/softlib2/software1/sc-windows/p2015029342/v154233/cp036919.exe). The download for the Windows Server 2016 version is here: https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX-c5e0ed82c4864327bc9b86442c#tab-history. For other versions, just google “iLO 3/4 Channel Interface Driver Server 2008 R2” for example.
You may have to reboot afterwards in order to finish the installation. Hopefully you are doing all of this on a new server so you won’t be interrupting anything important. Otherwise use a scheduled task to reboot the server at a time when no one will notice.
Next we will install the HPE System Management Homepage which will allow you to check the status of your server in the web browser at https://localhost:2381.
Use this link to download it and run the executable as an administrator: https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_b36fb879335a41d78ee5e99f7f
In order for the HPE tools to communicate with each other (and us!) we need to install the SNMP service. Microsoft describes the SNMP services as follows: “The Simple Network Management Protocol (SNMP) is used to configure remote devices, monitor network performance, audit network usage, and detect network faults or inappropriate access.” Exactly what we want, right?
To install the SNMP service, go to Server Manager, then click “Manage”, then “Add Roles and Features”. Choose “Role-based”, Next, select the local server, Next, Next, on the “Select Features” screen check the box next to “SNMP Service” and the one inside it, “SNMP WMI Provider”. You may be prompted to install “SNMP Tools”, too, if you don’t have it installed already. Get it, you’ll need it.
Click Next, then click Install.
Once the installation of SNMP service is finished, click the Start menu and type services.msc (or Windows key and R key on the keyboard and enter services.msc) to open the Services interface.
Find the SNMP Service and make sure it’s running.
Next, we will download and install the HP Insight Management Agents. Here is the link for the newest version at the time of this writing: https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_1b0b099098874404adcb5d1d51#tab1
Run the executable as and admin and click Install.
When the install completes, the “Management Agents” window will open.
When the install is finished the “HPE Management Agents” settings will open automatically. You may notice there is a button on the “SNMP Settings” tab that says: “Send Test Trap”. We will come back to this later. It won’t work until we take care a few other things so you can click cancel for now.
Now that we have the HPE agents installed we need to configure the SNMP service to collect information from the agents. Open Services.msc by searching in the Start menu or Windows key + R and entering Services.msc. Scroll down to “SNMP Services” and double-click on it. In the security tab, click Add button under “Accepted community names”. There are two Add buttons on this tab so click the top one. Select Community rights as “READ ONLY”. Type “Public” for the Community Name and click Add.
If you don’t have the Security tab close Services.msc and restart it.
Click the same Add button again and create another Community Name called “Private”, but this time make the Rights “READ WRITE”.
Now we will tell the SNMP service who to accept packets from. With the “Accept SNMP packets from these hosts” radio button selected, click on the lower Add button now and enter 127.0.0.1 as the IP address. This is the localhost so the SNMP service will only be listening to this server we are working on. Click Add and then Apply.
With this done, go to the Traps tab in SNMP Service properties. You will have to type directly into the drop-down list because it doesn’t populate itself. Type “public” and now the “Add to list” button will be clickable. Click it. The drop-down list now has an entry for “public”. Click the “Add” button lower down the Traps tab and enter 127.0.0.1 and then click the Add button.
Click Apply and Ok.
You may notice that you have several new services in your Services.msc window. Their names differ depending on the version you install, but they will be similar to:
HP Insight Event Notifier
HP Insight Foundation Agents
HP Insight NIC Agents
HP Insight Server Agents
HP Insight Storage Agents
HP Smart Array SAS/SATA Event Notification Service
HP System Management Homepage
HP WMI Storage Providers
The names pretty much tell you what they do, but you can click on them in Services.msc to get a more detailed description.
They should all be started except for the HP Insight Event Notifier service. It needs to be configured before it will start. Let’s do that now. In the Start menu go to “HP Management Agents” and run the “Event Notifier Config” as administrator. The instructions are right there in the wizard for you.
You can also click on the Events button to choose which events trigger the notications. You can have different events selected for different recipients. All of them are selected by default. When you have all of your recipients in there, click finish.
Keep in mind any time you want to change these settings you must open “Event Notifier Config” as an administrator by right-clicking. Otherwise the settings will all be blank like it was never set up in the first place.
Back in the Services.msc window right-click on “HP Insight Event Notifier” and then click Restart. You can also double-click on it and then click the Start button.
If the “HP Insight Event Notifier” service doesn’t start, check your event viewer and you may see an error that will help you. For example, if you try to start that service without first running through the “Event Notifier Config” you will see two errors:
If that’s the case, set up the “Event Notifier Config” and it should work.
Back in the Services.msc window find the “HP System Management Homepage” service. Right-click it and click “Restart”.
Now let’s open the HP System Management Homepage. There is probably a shortcut on your desktop, but if not, you will find it in the Start menu under “HPE Management Agents”. You can also type https://localhost:2381 into your browser. Log in to the homepage with an administrator account. If you are using a domain admin account enter it in the format of DOMAIN\username. If you’ve set everything up correctly you should see something like this:
To see some diagnostics, you can go to Logs > Integrated Management Log:
Explore this to see all of the nice stuff it tells you about your server. However, you can’t always be in here staring at it and waiting for problems to pop up, so we need to get the alert emails coming in. There are a couple of ways to send a test email. One is right here in the System Management Homepage.
The next part is different depending on the version of these programs you have.
To send a test email alert with the System Management Homepage, first click on Settings, then SNMP & Agent Settings.
In here you will find another view of the SNMP settings we saw in Services.msc. On the left side of the page you should find the “Send Test Trap” button. Click that and then click Ok when it asks you if you are sure. This next screenshot is from version 184.108.40.206 of the System Management Homepage. Other versions look a lot different and don’t give you as much control. Also, they are moving this functionality into the iLO starting with Gen 10 servers. More on that in another article.
Some versions of the Homepage have a different area where you can send a test indication, but it doesn’t send an email. It just sends a message to the event viewer. To try that out you can go to Settings, Test Indication, Send Test Indication. This is from version 220.127.116.11:
Here is an example of the “Test Indication” error that was sent to the event viewer:
The most reliable place to send a test email from is the Control Panel app they’ve given us as part of the HP Management Agent installation. This app seems to be present in all versions of these programs.
Open the Windows Control Panel and find “HP Management Agents”. Click on “View By” and choose “Large Icons” to find it easier. Right-click on it and run it as administrator.
In the first tab that opens you will see all of the types of agents that are working in the background. The important ones should already be set up for you. To send a test email alert click on the “SNMP Settings” tab.
You don’t need to change anything here either. Just click the “Send Test Trap” button. If the “Send Test Trap” button is greyed out, check to make sure the “HP Insight Foundation Agents” service is running in Services.msc and you are running “HP Management Agents” as administrator. If you see the following error after clicking “Send Test Trap”: “The Management Agents Remote Alerter agent has not finished initializing or is not enabled. The test trap was not sent.” then you are probably not running “HP Management Agents” as administrator.
If you’ve set everything up correctly you should get a message box like this: “The following trap has been sent” with the date and time.
And you should receive and email that says “The system has detected the following event:” with the date, time, server name, and the description of the error:
If you get that email, then congratulations! You’re all set to receive an email alert when something goes wrong.
If you don’t get an email you may have to add the server to your Exchange receive connectors or make some other change depending on how you are set up. Also, check the services in Services.msc to make sure they didn’t stop running for some reason.
Here is an example of a dead battery error:
You might want to try sending a test email once a month or so to make sure it’s all still working. Have fun!