Recently I was encouraged to find a way to monitor many of our appliances with our shiny new deployment of Microsoft System Center Operations Manager 2007 R2. Up until this point we had not used SCOM for SNMP monitoring little did I know the adventure I was about to embark on.
Requirements:
Log (SCOM Rule) various values from different SNMP appliances
Alert (SCOM Monitor) on various values from different SNMP appliances
Basic availability checks for all appliances
Receive and alert on SNMP TRAPS from all appliances
The Process : (no point in trying to skip to the solution)
The first thing I found was a lot of people in various blogs with a warning “Abandon hope all ye who enter here”. Usually I would save my ranting comments to the end but perhaps it’s best to give you a quick glimpse now. SCOM 2007 R2 is not exactly the pinacle of SNMP monitoring, but I will get into that more later 😉
In part 1 we will find a way to discover and differentiate between the different kinds of SNMP appliances so that we can target various monitors and rules at specific devices. No point is weighing down SCOM with monitoring all sorts of thinsg that don’t exist on a given device right?
Step 1: Install required tools
Here I have to start with a major and well deserved tip of the hat to 2 individuals without whom my adventures into SCOM and SNMP would have been long and likely fatal… or very short and almost painless (Hmmmm) either way, Thanks to Raphael Burry and his SNMP Discovery Provider for OpsMgr 2007 and Scott Vintinner with his Example SNMP Management Pack for SCOM 2007. Without these 2 pieces I would likely have left SCOM as an SNMP monitoring too altogether.
First download the sealed managed pack from Raphael rename it to a zip, break out the management pack and install it into you SCOM installation. This gives us the extensions to start writing out own custom management packs to start discovering different types of SNMP devices.
Second do your self a favor and download and install the iReasoning MIB browser and WireShark (formerly knows as Ethereal for those that missed the name change a while back) these tools will be invaluable shortly, trust me.
Step 2: Figure out how to identify devices
First we need to figure out what makes this device different from any other device in our network, this is where the iReasoning MIB Browser comes in handy. After you have configured your device with an SNMP community name and allowed traffic from the machine you are running the MIB Browser from it’s time to pick an OID. Start the MIB Browser. You can load manufacturer MIB’s if you want but we don’t need them here. Enter the IP address of the device, clear anything that is in the IOD field and use the operations dropdown to pick walk. Then click GO!
Edit : Daniel Morrison makes a good comment – you may also need to hit “Advanced” and enter the SNMP community you configured on teh device for the walk operation to work. The default value is Public. Just in caseyou missed it, you can see it below just between Address and OID.
Now you can go down the list of OID’s that you see and find something specific that will be unique to this type of device. For this example we are looking at an SG series GE UPS. In this case I am going to go with 1.3.6.1.2.1.1.2.0 that provides an answer of 1.3.6.1.4.1.818.1.100.1.1 with these noted it’s time to start editing our management pack.
Step 3: Your first custom SNMP discovering management pack!
Here again you will want a couple of tools to make life much easier. First XML Notepad 2007 a Microsoft tool for editing XML, handy for when the next tool does not expose what you want. Second SCOM Authoring Console 2007 R2, from the AuthoringConsole directory in the root of your download or SCOM 2007 R2 CD. This is probably on the website somewhere but what isn’t anymore?
Now you are ready to download Scott Vintinner’s Example Management pack, (EDIT: or the updated version here) this is not installed just used as a framework to build our own custom pack for the device in question. Make a copy and open it with notepad. The first thing to notice is right near the top, <ID>RBH.Ecosaire.AC.Management.Pack</ID> we need to change this to match whatever we are doing like <ID>GEUPS.Example.Management.Pack</ID> so I do a find and replace on the whole XML from “RBH.Ecosaire.AC.” to “GEUPS.Example.” then I save and close the file. At this point you have to rename the file to match the ID in this case GEUPS.Example.Management.Pack.xml
Now for the discovery bit. Open the XML with your preferred editor and look for <Discoveries> this section is all we plan to edit at this stage. Within <Discoveries> you will find
<SnmpVarBinds>
<SnmpVarBind>
<OID>1.3.6.1.2.1.1.2.0</OID>
<Syntax>0</Syntax>
<Value VariantType=”8″ />
</SnmpVarBind>
</SnmpVarBinds>
The key to this section is the OID this is what is queried to determine if this device is a GE UPS as defined by our management pack. A few lines further down you have another important section, we will start with a simple expression first.
<Expression>
<SimpleExpression>
<ValueExpression>
<XPathQuery>/DataItem/SnmpVarBinds/SnmpVarBind[OID=’1.3.6.1.2.1.1.2.0′][1]/Value</XPathQuery>
</ValueExpression>
<Operator>Equal</Operator>
<ValueExpression>
<Value Type=”String”>1.3.6.1.4.1.8072.3.2.10</Value>
</ValueExpression>
</SimpleExpression>
</Expression>
A few important things to notice in this section. First on the line with XPathQuery you see an OID, if you changed the OID in the <SnmpVarBind> section above you need to change it here as well as this is the pointer to the variable that was read and it needs to match for the compare or who knows what will happen. I of course know one thing that will happen and that is that you will never discover anything. So I guess with further thought I do know what will happen. The second thing to note is <Operator>Equal</Operator> so this is a simple X = Y expression, we will look at another option in a second. Third <Value Type=”String”>1.3.6.1.4.1.8072.3.2.10</Value> this is telling us that the data type of this variable is a string, so you can’t try and match based on > or anything like that, and the value we are looking for is 1.3.6.1.4.1.8072.3.2.10, of course this is the example value and not the value we determined above so I will replace it with 1.3.6.1.4.1.818.1.100.1.1 and save the XML. Optionally You can look for <Interval>3600</Interval> this is the number of seconds between discoveries. This can also be altered once the MP has been imported but unless you like to wait (in which case you will get along great with SCOM) you may want to reduce this now for testing. Try not to forget to change it back later 😉
Now if a simple expression is not good enough you may need a regular expression here is an example for another device I recently used. To be honest I am still looking for a good source of documentation on all the options for e regular expression if anyone knows a good one. (EDIT – Thanks Steve for pointing out the document Regular expression support in SCOM 2007.docx from the OpsManJam website. )
Here is the sample
<Expression>
<RegExExpression>
<ValueExpression>
<XPathQuery>/DataItem/SnmpVarBinds/SnmpVarBind[OID=’1.3.6.1.2.1.1.1.0′][1]/Value</XPathQuery>
</ValueExpression>
<Operator>MatchesRegularExpression</Operator>
<Pattern>^.*SensorHawk .*$</Pattern>
</RegExExpression>
</Expression>
With the above section of XML if the txt SensorHawk appears in the result then it’s considered a match.
Edit: Ben needed a discovery that would detect 2 different kinds of devices. Together we found the following appears to work best.
<Expression>
<Or><Expression>
<RegExExpression>
<ValueExpression>
<XPathQuery>/DataItem/SnmpVarBinds/SnmpVarBind[OID=’1.3.6.1.2.1.1.1.0′][1]/Value</XPathQuery>
</ValueExpression>
<Operator>MatchesRegularExpression</Operator><Pattern>^.*RICOH .*$</Pattern>
</RegExExpression>
</Expression>
<Expression>
<RegExExpression>
<ValueExpression>
<XPathQuery>/DataItem/SnmpVarBinds/SnmpVarBind[OID=’1.3.6.1.2.1.1.1.0′][1]/Value</XPathQuery>
</ValueExpression>
<Operator>MatchesRegularExpression</Operator><Pattern>^.*Canon .*$</Pattern>
</RegExExpression>
</Expression>
</Or>
</Expression>
As awesome as Scott Vintinner’s example one thing I believe it lacks is a view in the console so you can see whats going on. We need to add just a bit of XML to the existing pack, to make this easy I have added it to base pack using the RBH.Ecosaire.AC naming so you can just edit it all at once if you prefer. Download updated management pack example Here we have added a section just after the end of </Monitoring>
<Presentation>
<Views>
<View ID=”RBH.Ecosaire.AC.Management.Pack.AlertView” Accessibility=”Internal” Enabled=”true” Target=”RBH.Ecosaire.AC.Management.Pack.SNMPDevice” TypeID=”SC!Microsoft.SystemCenter.AlertViewType” Visible=”true”>
<Category>Custom</Category>
<Criteria />
</View>
<View ID=”RBH.Ecosaire.AC.Management.Pack.EventView” Accessibility=”Internal” Enabled=”true” Target=”RBH.Ecosaire.AC.Management.Pack.SNMPDevice” TypeID=”SC!Microsoft.SystemCenter.EventViewType” Visible=”true”>
<Category>Custom</Category>
<Criteria />
</View>
<View ID=”RBH.Ecosaire.AC.Management.Pack.PerformanceView” Accessibility=”Internal” Enabled=”true” Target=”RBH.Ecosaire.AC.Management.Pack.SNMPDevice” TypeID=”SC!Microsoft.SystemCenter.PerformanceViewType” Visible=”true”>
<Category>Custom</Category>
<Criteria />
</View>
<View ID=”RBH.Ecosaire.AC.Management.Pack.StateView” Accessibility=”Internal” Enabled=”true” Target=”RBH.Ecosaire.AC.Management.Pack.SNMPDevice” TypeID=”SC!Microsoft.SystemCenter.StateViewType” Visible=”true”>
<Category>Custom</Category>
<Criteria />
</View>
</Views>
<Folders>
<Folder ID=”RBH.Ecosaire.AC.Management.Pack.ViewFolder” Accessibility=”Internal” ParentFolder=”NetLib!Microsoft.SystemCenter.NetworkDevice.AllDevices.ViewFolder.Root” />
</Folders>
<FolderItems>
<FolderItem ElementID=”RBH.Ecosaire.AC.Management.Pack.AlertView” Folder=”RBH.Ecosaire.AC.Management.Pack.ViewFolder” />
<FolderItem ElementID=”RBH.Ecosaire.AC.Management.Pack.EventView” Folder=”RBH.Ecosaire.AC.Management.Pack.ViewFolder” />
<FolderItem ElementID=”RBH.Ecosaire.AC.Management.Pack.PerformanceView” Folder=”RBH.Ecosaire.AC.Management.Pack.ViewFolder” />
<FolderItem ElementID=”RBH.Ecosaire.AC.Management.Pack.StateView” Folder=”RBH.Ecosaire.AC.Management.Pack.ViewFolder” />
</FolderItems>
</Presentation>
And a few display strings that will form our text labels for the above entries, these go in the <LanguagePacks> section just before </DisplayStrings>
<DisplayString ElementID=”RBH.Ecosaire.AC.Management.Pack.AlertView”>
<Name>Alerts</Name>
</DisplayString>
<DisplayString ElementID=”RBH.Ecosaire.AC.Management.Pack.EventView”>
<Name>Events</Name>
</DisplayString>
<DisplayString ElementID=”RBH.Ecosaire.AC.Management.Pack.PerformanceView”>
<Name>Performance View</Name>
</DisplayString>
<DisplayString ElementID=”RBH.Ecosaire.AC.Management.Pack.StateView”>
<Name>State View</Name>
</DisplayString>
<DisplayString ElementID=”RBH.Ecosaire.AC.Management.Pack.ViewFolder”>
<Name>Ecosaire AC</Name>
</DisplayString>
Now all you have to do is install your new MP and you should see in the SCOM monitoring console
With luck now you can see your base device management pack and may have even have enough good karma to see some devices. Of course if all you have done is followed my instructions you still won’t have anything because there is one last step.
Step 4: You need to discover the device in SNMP. This is done via the operations console, in teh Administration tab. Right click on “Device Management” and run the “Discovery Wizard”
Simple enough, select network devices and select next.
Here enter the IP or range of IP’s that your devices user, make sure to enter the community name you configured on the device, and drop the SNMP version down to 1, unless of course you know your device is V2. Select the management server that you want to handle traps, monitors and rules and click discover.
If all goes well a couple of minutes later you will get a screen showing the devices that have been doscovered, check the box(es) of the ones you want to be managed ckick finish and you are done. Then wait a while and they should start showing up in the management console. If things didn’t go well there is likely a problem with either the community name or the SNMP configuration on the device allowing your root management server to contact the device using SNMP. Best to configure the device to send SNMP traps to all you management servers and allow SNMP read-only from all of your management servers.
Here are the completed XML files for a few of the MP’s I created if they are of use to you. Download and rename to .XML
GEUPS.GreaterThen.Five.Management.Pack
GEUPS.Single.Phase.Management.Pack
RBH.Ecosaire.AC.Management.Pack
Stay tuned for Part 2 where we will look into createing rules and monitors for the discovered devices.
Nice document…and funny. Is part 2 available yet? Thanks again, it help a lot.
Part 2 and 3 are in progress, should be out before the end of the month.
Great article. One small suggestion – for step 2, you may also need to hit “Advanced” and enter the SNMP community for the walk operation to work.
Good point, I have updated the entry with your comment.
Thanks.
No worries. I have configured SNMP performance collection rules but I can’t get them to collect any data, and SCOM doesn’t seem to do any logging around SNMP actions. So… looking forward to part 2 😀
It’s almost ready, I will email you as soon as we post.
Excellent write up there Scott! Now you’ve finished unwrapping your Christmas presents and devouring the Christmas turkey, when’s Part 2 likely to turn up? 🙂
Sadly Christmas is but a distant memory now, I am very sorry for the delay. I really do intend to get part 2 done soon. The project I am working really is at a fever pitch right now but I will find a way somewhere 🙂
Hello,
Can you please suggest how to configure monitoring for printers.
I have the printer discovered but the traps arnt being proccesed in scom.
Thank you in advance 🙂
If you have managed to get as far as getting the device discovered you are almost there.
Have a look at part 3 in the series
https://www.blackops.ca/cms/blog/?p=102
Initially create a wildcard trap rule and see if that works, then you can get more specific if it does.
If not make sure that your printer is sending the traps to the SCOM Management server that you used during the discovery. Then have a look using wireshark to make sure traps are arriving (in part 3) and confirm the snmp services are installed on the management server receiving the traps.
Let me know how it goes for you or is you need more assistance.
Great Article, I have one question:
we have a device that contain different modules attached to it. the only way to get those modules is by doing SNMPWALK
can we modify the XML to do SNMP Walk on OID which will help in discovering those modules?
Regards
I don’t know of any way to adjust the XML to actually do an SNMP Walk to detect “sub” devices.
In the past where I needed to do this I detected the parent device using the detection process listed in these articles. Once I had the parent device detected I created a series of monitors and rules targeting the specific OID’s of the child devices. I needed to create a monitor for each possible child module for every expansion port and then disable the ones I didn’t need.
I understand of course if there are a lot of options for the sub modules it can be unrealistic to create all the combinations and the disable all the monitors and rules that don’t actually apply on a device by device basis.
If this is not clear or you need some help use the contact form to send me an email.
Good Luck.
Great writeup!
I made 2 MP’s with it. 🙂
@Bas: Could you please send me the management pack that you created?
Thanks in advance
I added 3 completed XML files for management packs I used this technique to create.
They are specific to the equipment I was working on and I never really tried to create an MP that anyone could just install.
You should be able to use these and the other files linked in the first article to recreate the complete pack.
Let me know if this works for you.
Hi all
i am willing to send my SCOm alarms to another aggregation system but the problem is i can not find the Scom2007R2 MIb file (so as to define the traps at the receiving end) ? can any one advice for this please/thanks
I have never personally sent traps from SCOM to another receiver I am not sure exactly how you are doing that.
you may want to have a look at http://social.technet.microsoft.com/Forums/en-US/operationsmanagerextensibility/thread/122f656a-5109-452a-ba1a-a17c89d347d6/
Depending on how you are transmitting the traps I would hope that it would pass on the OID from the initial alert, then you could use the MIBS form the source and not have to create your own.
Let me know if you have any more details about what you are monitoring, how and how you are sending the traps and I will see if I can help.
Hi Scott
Wondering if the above would work for scom 2012 too. I had been trying to create a mp to discover my isilon storages. But the discovery is not happening. Is there something different that I should do with SCOM 2012. Should I configure the SNMP service/SNMP trap in someway. I am out of ideas and any help would be greatly useful.
Shawn appears to have managed the discovery but was having trouble with the trap alerts in part 2.
I don’t have a 2012 to test with so I am not sure.
Hi Scott,
Enjoyed reading this as I need to get a lot of SNMP monitoring going. I had a problem with some of the sections of the MP under Presentation and needed to remove some lines in order to get my MP imported, but it is all there now. However; one question: even after my discovery (which has worked) should I see anything under the State View as your bitmap shows or will this get populated when I have continued through parts 2 and 3?
I figured no point continuing if I had a problem at this stage.
If I recall all you will see at that point is that it has been discovered. You need to do the next parts to get it monitored and have a status.
Good luck.
If you don’t mind post the lines you needed to remove in case it help others, thanks.
No panic, I have revisited the MP I created and started again; this time get no errors.
Originally, the lines I took out were down the bottom section and were:
Discover Ecosaire Air Conditioner
Ecosaire Air Conditioner SNMP Device
The import would error with these lines so I obviously hadn’t correctly edited all the text to change it to the devices I am discovering.
Now moving on to part 2!
Glad to hear it, Good Luck.