ServiceNow IntegrationHub ETL

Platform

runZero Platform integrates with ServiceNow Configuration Management Database (CMDB) through a runZero JSON endpoint, with asset data formatted as CMDB Configuration Items (CIs). This integration brings runZero data into ServiceNow, allowing for specific fields and CI class mappings to be fine-tuned from the ServiceNow console.

A basic import can be performed using ServiceNow IntegrationHub ETL. ETL allows you to create transform maps via point-and-click operations without programming. It processes incoming data via the ServiceNow Robust Transform Engine (RTE), and ensures that the Identification and Reconciliation Engine (IRE) is used to integrate the data into your existing ServiceNow CMDB instance.

For more advanced imports, you can of course develop transform maps manually, and import the data that way.

This document will briefly cover the process of setting up ServiceNow IntegrationHub ETL to fetch data from runZero via REST API.

Before you begin

• Because ServiceNow is so customizable, it is impossible to make integration a “one-click” process. At a minimum, you will need to spend time deciding which CI classes you want to use, and how you want to map data to their fields.

• You will definitely want to test the import process in a development instance of ServiceNow. You can obtain one by signing up for ServiceNow’s developer program.

• IntegrationHub ETL may need to be installed from the ServiceNow store.

• ServiceNow ITOM is licensed based on the number of CIs in your CMDB, regardless of whether they were discovered by ServiceNow Discovery or by another tool such as runZero. You will need a subscription to ITOM Visibility or ITOM Discovery, with enough Subscription Units to cover the number of CIs in your organization.

Set up an ETL import

To set up an ETL import, you’ll need to:

• Install IntegrationHub ETL
• Set up runZero as a discovery source
• Set up credentials
• Set up a connection and credential alias
• Set up a connection
• Set up a data source
• Create a new data flow action
• Set up an ETL import

Install IntegrationHub ETL

For a development ServiceNow instance, you will need to install the IntegrationHub ETL application:

1. Go to System Applications > All Available Applications > All.
2. Search for IntegrationHub ETL by name, and wait until it appears.

3. Click Install on the info page, then Install again on the confirmation.

ServiceNow will automatically install Integration Commons for CMDB and Integration Studio API as well. Once installation is complete, you will need to shift-reload the browser page before IntegrationHub ETL shows up in the left navigator.

Set up runZero as a discovery source

The first step is to set up a discovery source to let you flag CIs which were added from runZero data.

Navigate to System Definition > Choice Lists in ServiceNow. Search for the table cmdb_ci, and the element discovery_source. Assuming there isn’t one for runZero, click New to create a new choice.

In the Table search box, enter [cmdb_ci] (including the square brackets), and click Configuration Item [cmdb_ci]. The remaining required values are:

Click Submit to create the new choice.

Set up credentials

You will need a runZero organization API key. These are set up via the Organizations page in runZero. You can click on an organization, scroll down to the bottom of the page, and generate a new API key or copy an existing one. The API key looks like a string of hexadecimal digits.

Once you have your API key, navigate to Connections & Credentials > Credentials in ServiceNow, click New, and select API Key as the type of credential to create.

Give the new credential a sensible name like ‘runZero export API key’, paste in the hex string as the API Key value, then click Submit.

Set up a connection and credential alias

Next, you will want to set up a connection and credential alias. To do this, go to Connections & Credentials > Connection & Credential Aliases and click on New. Provide the necessary information:

The defaults should be OK for the other fields.

Set up a connection

Now it’s time to set up the actual HTTP connection based on the credentials and alias you’ve created. Navigate to Connections & Credentials > Connections, click New, and choose to set up an HTTP(S) Connection.

Enter the required field values:

Note that the connection alias name will have had spaces and other punctuation replaced with underscores.

Set up a data source

Now you should be ready to start setting up a data source for the runZero data.

Go to System Import Sets > Administration > Data Sources and click New.

Some suitable field values are:

Note that the import set table name is required to start with u_. There shouldn’t be any arrays in the exported data, but you can check the ‘Discard Arrays’ option to be safe.

With the basic info in place, check towards the top right of the form next to the Request action field. Clicking the button to the right of that field will launch the ServiceNow Flow Designer to create a new data flow action.

Create new data flow action with the flow designer

The flow designer will prompt you to name your new action, so give it a descriptive name like “Fetch runZero assets”.

Once you confirm the action name and other basic info, you should find yourself looking at an action outline with three steps:

Click on REST step on the left to edit the REST action details.

Under Headers, you need to click to add a new header. The header name is Authorization, and for the value you should type in Bearer followed by a space. You can then drag the lozenge labeled Credential Value from the right side of the screen into the Value box, and it should show up as per the screenshot below:

Note that you can build the resource path using a similar method and the Attachment Name lozenge, but in this case we suggest hard-coding it as it doesn’t need to change for different attachment names.

At this point it’s probably a good idea to use the Test button at the top of the screen to test the action. You can provide a fake value for the attachment name and click Run Test. ServiceNow should (eventually) say:

Your test has finished running. View the action execution details.

To do so, close the dialog and click the Executions button to open the Operation List tab. Click the link to open the most recent test operation into another tab. Under Output Data, make sure you got status_code 200 (HTTP OK) from runZero. If you get an error in the 400 range, there’s probably an error with the Authorization header or your API key.

You might optionally want to check the Response Payload Size by twisting the disclosure triangle next to Steps.

If the action is working, close both the Operation tabs to go back to the “Fetch runZero asset data” action, and click the Publish button.

Now you can close the Flow Designer tab in your browser and go back to setting up the ServiceNow data source.

Back on the data source page, you should now be able to use the ‘Request action’ field to add the request action you just created, and save the data source.

Set up an ETL import

Go to Configuration > IntegrationHub ETL and click New to create a new set of transform maps. The ETL tool leads you through the setup in numbered stages.

In section 1, click on Import Source Data and Provide Basic Details.

Note that the name is used for transform map names.

Once you’ve filled out the basic information, clock Save, and also Mark as Complete so that ServiceNow will allow you to proceed to step 2.

In section 2, click Preview and Prepared Data. The IntegrationHub ETL tool should load in the first 10 rows of your runZero data.

At this point, you can proceed with the normal process for importing data via IntegrationHub ETL. You can transform fields, decide which asset types become which classes of CI, and map fields in the normal way. A good introduction to the rest of the process is the ServiceNow Knowledge2020 course “Use IntegrationHub ETL to Import Data”.

Things to know about runZero export data

MAC addresses and network adapters

ServiceNow supports modeling computers with multiple network cards via the Network Adapter CI class. Unfortunately, the Network Adapter CI requires a name for the network adapter. runZero has no way to obtain that information, and it differs between OSs and OS versions.

One option is to set a standard name for all Network Adapter CIs. You can then use ETL’s lookup tools or Glide Lookups to search existing computer data by MAC address and avoid duplicates.

Specific fields

Here are some field-by-field notes on the data in runZero’s ServiceNow export.

Note that some fields can contain multiple values, in which case they are separated with semicolons.

• asset_id: This is a universally unique ID (UUID) for the runZero asset. It will remain the same across runZero scans, so it can be used as the source native key in ServiceNow.

• type: This is the runZero asset type.

• sys_class_name: This is a suggested ServiceNow CI class that might be suitable for the asset. Not all assets will have a sys_class_name value supplied, and you might disagree with some of the suggestions. We hope to improve the suggestions over time.

• os_vendor: The vendor responsible for the OS running on the asset.

• os: The specific OS product running.

• os_version: The version of the OS product, if it could be determined.

• manufacturer: The manufacturer or vendor of the primary hardware.

• model: The specific product. You will most likely want to use a Glide Lookup or the ETL field mapping features to look up the manufacturer and model, and convert them to references to the manufacturer and product tables.

• ip_address: The primary IP address the asset was detected at.

• addresses_scope: A list of all of the IP addresses of the asset that were directly detected by scanning.

• addresses_extra: Additional IP addresses that we infer the asset could have, but which haven’t been confirmed by direct scan.

• mac_address: The primary MAC address for the asset.

• mac_manufacturer: The manufacturer assigned to the MAC address, generally the manufacturer of the network hardware.

• newest_mac_age: MAC address allocations have dates associated with them. This field has the approximate allocation date of the newest MAC address found associated with the asset. This can be used to provide an approximate age for the asset.

• macs: A list of all MAC addresses found for the asset.

• mac_vendors: A list of all the MAC vendors found for the asset.

• names, name: For convenience we provide a single-value name field, as well as names listing all of the identified names.

• last_updated: This is when the asset record in runZero was last updated. It can be used as an indication of data recency in ServiceNow.

Useful links

• ServiceNow CMDB developer information

• CMDB schema information

• ServiceNow Knowledge2020 course: Use IntegrationHub ETL to Import Data