Issue
Troubleshooting SNMP communications issues between Data Center Expert and APC Network Management Card Devices.
Product Line
StruxureWare Data Center Expert (DCE)
- Basic Appliance (AP9465)
- Standard Appliance (AP9470)
- Enterprise Appliance (AP9475)
- Virtual Appliance (AP94VMACT)
StruxureWare Central (SWC)
InfraStruXure Central (ISXC)
APC Network Management Card (AP9617, AP9618, AP9619, AP9630, AP9631, AP9635)
Environment
StruxureWare Data Center Expert (all versions)
StruxureWare Central (all versions)
InfraStruXure Central (all versions)
APC Network Management Cards
Cause
There are a number of reasons StruxureWare Data Center Expert (DCE) may not be able to communicate with a device. Lost communication usually occurs due to incorrect SNMP settings or SNMP being blocked on the network between the server and the APC device.
Resolution
The steps below will help resolve lost communication issues to APC Devices. It is recommended to check if your device has the latest firmware available. See Video: Data Center Expert | How do I mass upgrade APC and NetBotz appliance firmware? For APC devices. For third party devices (includes some APC devices that use alternative NMCs), resolution steps can be found within knowledge base Data Center Expert | Troubleshooting SNMP Lost Communication to Third Party Devices . If you do not know if the device is APC or Third-party, proceed below to determine.
Determining if your Device is APC or Third-Party
1) Log into the DCE Desktop Client using an Administrator account.
2) Navigate to the Monitoring perspective (button) and select the device group in the left-pane that contains the device in lost communication.
3) Right-click on the device within the Device View pane (tab).
- If there is an option for "APC SNMP Device Configuration..." proceed to SNMP Lost Communication Troubleshooting.
- If there is no option for "APC SNMP Device Configuration..." navigate to knowledge base Data Center Expert | Troubleshooting SNMP Lost Communication to Third Party Devices .
SNMP Lost Communication Troubleshooting
1) Within the Device View pane take note of the IP Address of the device that is in lost communication.
2) Navigate to: Device Menu > SNMP Device Communication Settings > Device Scan Settings.
3) Search for the Device's IP Address and then look within the Protocol column for that device to determine SNMP version.
4) If the device is in intermittent lost communication, proceed to step 5. If the device is in persistent lost communication, proceed to step 6.
NOTE: Intermittent lost communication is when the device loses communication to DCE and then re-establishes communication multiple times.
NOTE: Persistent lost communication is when a lost communication alert for the device is constant within DCE and is not clearing and reoccurring.
5) Intermittent lost communication occurs when DCE does not receive a response back for the SNMP request to the device within a set timeframe. This usually occurs due to network traffic issues or end-device processing issues. The usual resolution to this issue is to increase timeouts and retries for communication before DCE will report a lost communication alarm:
a) Check the box next to the device searched for in step 3.
b) Click the Edit Device Scan Settings... button.
c) Increase the Timeouts (seconds) and Retries.
NOTE: If having regular intermittent lost communications, it is recommended to increase the values to 10 second Timeouts and 5 Retries.
d) Click OK to save the changes.
NOTE: If the issue returns after increasing the timeouts and retries, the issue is commonly network related or end-device related. Contact your network team and/or technical support for assistance.
6) Persistent lost communication to devices occurs when DCE does not receive any responses back to SNMP requests to the APC Device. This usually occurs due to incorrect SNMP Settings within DCE or the Device. This may also occur due to changes on the network that may block the SNMP traffic from DCE to the device or from the device to DCE. If your device uses SNMPv1, as determined in step 3, proceed to SNMPv1 Persistent Lost Communication Troubleshooting. If your device is using SNMPv3, as determined in step 3, proceed to SNMPv3 Persistent Lost Communication Troubleshooting.
SNMPv1 Persistent Lost Communication Troubleshooting
1) Navigate to the Monitoring perspective and select the device group that has the device in lost communication.
2) Locate the device that is in lost communication and take note of the Application Version listed within the Device View pane.
NOTE: Steps below have different locations for settings depending on the application version of the NMC.
3) Right-click on the device within the Device View pane and select Launch to Device. This will load the web interface of the APC NMC.
NOTE: If you are unable to launch to the device, verify the Device Launch Settings and use DCE's Ping Utility.
4) Log into the web interface. Auto-login may occur if the correct credentials are defined within DCE.
NOTE: For instructions on configuring auto-login see knowledge Video: Data Center Expert | How do I configure auto-login when launching to APC devices? .
5) Navigate to the SNMP settings to determine if SNMPv1 is set to enabled.
- Application Version 6.X: Configuration menu > Network > SNMPv1 > Access > Enabled [checked]
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv1 > Access > Enable SNMPv1 access [checked]
6) Navigate to the SNMP community string settings within the APC NMC.
- Application Version 6.X: Configuration menu > Network > SNMPv1 > Access Control
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv1 > Access Control
7) Data Center Expert requires the ability to be able to read and write data via SNMP to the APC NMC. If only Read access is provided, Priority Scanning will not function. Ensure that there is an entry for Read and another for Write that are activated.
NOTE: Instead of two entries, one entry with the Access Type of Write+ will work for both Read (Get) and Write (Set) requests.
8) Ensure that the NMS IP/Host Name field applies to DCE. For a description of what your entry within this field means, click on Help in the upper-right corner of the NMC while on the SNMP Access page or while editing a specific SNMP Access line.
9) Take note of what the enabled Read/Write community strings are or the Write+ community string is.
10) DCE Desktop Client > Device menu > SNMP Device Communication Settings > Device Scan Settings > Search for device > Check Box on device line item > Edit Device Scan Settings.
NOTE: If selecting multiple devices, the settings that are currently displayed on the Edit Device Scan Settings screen will be set for communication between DCE and the Devices. If the devices each have different settings, this will cause lost communication to any devices that do not match these new settings that are applied.
11) The community strings recorded in step 9 should match. If they do not match, either the community strings within the Device Scan Settings or the community strings within the APC NMC SNMP Access settings must be updated.
STOP! Once the community strings are updated, navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. If the lost communication alarm does not clear after one (1) minute, proceed to step 12.
12) Increase the timeouts to 10 seconds and retries to 5. This will allow the device additional time to respond before DCE reports a lost communication with the device.
STOP! Navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. If the lost communication alarm does not clear after one (1) minute, proceed to step 13.
13) Log into the APC NMC web interface and Navigate to the Event Log. Look for any SNMP error messages, for which there are two scenarios:
- Unauthorized user attempting to access SNMP interface
- An SNMP settings is incorrect on the APC NMC or within the Device Scan Settings within DCE. Verify that the following settings are correct/matching between the systems:
- NMC SNMPv1 Access is set to enabled.
- NMC SNMPv1 Access Control settings (A&B OR C).
a) Access Type Read, NMS IP / Host Name, & Community Name
b) Access Type Write, NMS IP / Host Name, & Community Name
c) Access Type Write+, NMS IP / Host Name, & Community Name
- Device Scan Settings within DCE for the device match the community strings defined within the APC NMC.
- Device Scan Settings within DCE for the device have increased timeouts and retries.
- No SNMP Error Messages
- Traffic is blocked on the network (DCE>NMC or NMC>DCE)
- To test if traffic is getting to the NMC, edit the device scan settings within DCE and alter the community string to be incorrect. Request a device scan by right-clicking on the device within Device View. After about a minute, check the APC NMC Event log again for error messages. If there is now an error message for "Unauthorized user attempting to access SNMP Interface from XXX.XXX.XXX.XXX", SNMP traffic is able to get to the device. This would indicate that traffic may be blocked on the network from the NMC to DCE. If no error is present after this test, traffic is not getting across the network from DCE to the NMC.
14) If the tests above concluded that traffic is blocked on the network, work with your networking team to resolve the issue. Contact technical support if the issue still persists.
SNMPv3 Persistent Lost Communication Troubleshooting
1) Navigate to the Monitoring perspective and select the device group that has the device in lost communication.
2) Locate the device that is in lost communication and take note of the Application Version.
3) Right-click on the device within the Device View pane and select Launch to Device. This will load the web interface of the APC NMC.
4) Log into the web interface. Auto-login may occur if the correct credentials are defined within DCE.
NOTE: For instructions on configuring auto-login see knowledge Video: Data Center Expert | How do I configure auto-login when launching to APC devices? .
5) Navigate to the SNMP settings to determine if SNMPv3 is set to enabled.
- Application Version 6.X: Configuration menu > Network > SNMPv3 > Access > SNMPv3 Access Enable [checked]
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv3 > Access > Enable SNMPv3 access [checked]
6) Navigate to the SNMPv3 User Profile within the APC NMC.
- Application Version 6.X: Configuration menu > Network > SNMPv3 > User Profiles
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv3 > User Profiles
7) Click into the applicable profile and ensure that the fields are filled out with your specific configuration settings. Information for each of the fields, including character limitations, can be found by navigating to the help link within the upper-right of the APC NMC interface. Take note of all of the settings as they will need to be entered into DCE Device Scan Settings for the device.
6) Navigate to the SNMPv3 Access Control settings within the APC NMC.
- Application Version 6.X: Configuration menu > Network > SNMPv3 > Access Control.
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv3 > Access Control.
7) Ensure that the specific user name that is used for DCE SNMPv3 communication has the Access Enabled and an applicable NMS IP/Host Name entry. To edit an entry, click on the User Name hyperlink.
NOTE: Additional information and descriptions of the fields, including what to enter within the NMS IP/Host name field can be found within the Help link in the upper-right of the APC NMC web interface.
8) DCE Desktop Client > Device menu > SNMP Device Communication Settings > Device Scan Settings > Search for device > Check Box on device line item > Edit Device Scan Settings.
9) The Authentication Passphrase, Privacy Passphrase, and protocols should match the ones recorded in step 7. Note that the Passphrase fields will be protected, so they will need to be reentered for verification. Do not copy and paste any fields as hidden characters or spaces my paste into the fields.
STOP! Navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. If the lost communication alarm does not clear after one (1) minute, proceed to step 10.
10) Log into the APC NMC web interface and change the SNMPv3 User Profile to simple settings, for testing purposes only, to:
NOTE: Instructions on navigating to these settings can be found in steps 6-7
- User Name: DataCenterExpert
- Authentication Passphrase: DataCenterExpert
- Privacy Passphrase: DataCenterExpert
- Authentication Protocol: None
- Privacy Protocol: None
11) Change the Device Scan Settings within DCE to match the new settings on the NMC, set in step 10.
DCE Desktop Client > Device > SNMP Device Communication Settings > Device Scan Settings > Search for device > Check Box on device line item > Edit Device Scan Settings.
STOP! Navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. This usually will establish communication, and is only done for testing and troubleshooting purposes. If the lost communication alarm does not clear, reach out to your local technical support for assistance. If the lost communication alarm clears, proceed to step 12.
12) Change the back to the intended settings within DCE and the NMC, ONE at a time to determine what may be causing the communication issue. Make any necessary changes to the value that is causing issues, ensuring to update it within DCE and the APC NMC.
Troubleshooting SNMP communications issues between Data Center Expert and APC Network Management Card Devices.
Product Line
StruxureWare Data Center Expert (DCE)
- Basic Appliance (AP9465)
- Standard Appliance (AP9470)
- Enterprise Appliance (AP9475)
- Virtual Appliance (AP94VMACT)
StruxureWare Central (SWC)
InfraStruXure Central (ISXC)
APC Network Management Card (AP9617, AP9618, AP9619, AP9630, AP9631, AP9635)
Environment
StruxureWare Data Center Expert (all versions)
StruxureWare Central (all versions)
InfraStruXure Central (all versions)
APC Network Management Cards
Cause
There are a number of reasons StruxureWare Data Center Expert (DCE) may not be able to communicate with a device. Lost communication usually occurs due to incorrect SNMP settings or SNMP being blocked on the network between the server and the APC device.
Resolution
The steps below will help resolve lost communication issues to APC Devices. It is recommended to check if your device has the latest firmware available. See Video: Data Center Expert | How do I mass upgrade APC and NetBotz appliance firmware? For APC devices. For third party devices (includes some APC devices that use alternative NMCs), resolution steps can be found within knowledge base Data Center Expert | Troubleshooting SNMP Lost Communication to Third Party Devices . If you do not know if the device is APC or Third-party, proceed below to determine.
Determining if your Device is APC or Third-Party
1) Log into the DCE Desktop Client using an Administrator account.
2) Navigate to the Monitoring perspective (button) and select the device group in the left-pane that contains the device in lost communication.
3) Right-click on the device within the Device View pane (tab).
- If there is an option for "APC SNMP Device Configuration..." proceed to SNMP Lost Communication Troubleshooting.
- If there is no option for "APC SNMP Device Configuration..." navigate to knowledge base Data Center Expert | Troubleshooting SNMP Lost Communication to Third Party Devices .
SNMP Lost Communication Troubleshooting
1) Within the Device View pane take note of the IP Address of the device that is in lost communication.
2) Navigate to: Device Menu > SNMP Device Communication Settings > Device Scan Settings.
3) Search for the Device's IP Address and then look within the Protocol column for that device to determine SNMP version.
4) If the device is in intermittent lost communication, proceed to step 5. If the device is in persistent lost communication, proceed to step 6.
NOTE: Intermittent lost communication is when the device loses communication to DCE and then re-establishes communication multiple times.
NOTE: Persistent lost communication is when a lost communication alert for the device is constant within DCE and is not clearing and reoccurring.
5) Intermittent lost communication occurs when DCE does not receive a response back for the SNMP request to the device within a set timeframe. This usually occurs due to network traffic issues or end-device processing issues. The usual resolution to this issue is to increase timeouts and retries for communication before DCE will report a lost communication alarm:
a) Check the box next to the device searched for in step 3.
b) Click the Edit Device Scan Settings... button.
c) Increase the Timeouts (seconds) and Retries.
NOTE: If having regular intermittent lost communications, it is recommended to increase the values to 10 second Timeouts and 5 Retries.
d) Click OK to save the changes.
NOTE: If the issue returns after increasing the timeouts and retries, the issue is commonly network related or end-device related. Contact your network team and/or technical support for assistance.
6) Persistent lost communication to devices occurs when DCE does not receive any responses back to SNMP requests to the APC Device. This usually occurs due to incorrect SNMP Settings within DCE or the Device. This may also occur due to changes on the network that may block the SNMP traffic from DCE to the device or from the device to DCE. If your device uses SNMPv1, as determined in step 3, proceed to SNMPv1 Persistent Lost Communication Troubleshooting. If your device is using SNMPv3, as determined in step 3, proceed to SNMPv3 Persistent Lost Communication Troubleshooting.
SNMPv1 Persistent Lost Communication Troubleshooting
1) Navigate to the Monitoring perspective and select the device group that has the device in lost communication.
2) Locate the device that is in lost communication and take note of the Application Version listed within the Device View pane.
NOTE: Steps below have different locations for settings depending on the application version of the NMC.
3) Right-click on the device within the Device View pane and select Launch to Device. This will load the web interface of the APC NMC.
NOTE: If you are unable to launch to the device, verify the Device Launch Settings and use DCE's Ping Utility.
4) Log into the web interface. Auto-login may occur if the correct credentials are defined within DCE.
NOTE: For instructions on configuring auto-login see knowledge Video: Data Center Expert | How do I configure auto-login when launching to APC devices? .
5) Navigate to the SNMP settings to determine if SNMPv1 is set to enabled.
- Application Version 6.X: Configuration menu > Network > SNMPv1 > Access > Enabled [checked]
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv1 > Access > Enable SNMPv1 access [checked]
6) Navigate to the SNMP community string settings within the APC NMC.
- Application Version 6.X: Configuration menu > Network > SNMPv1 > Access Control
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv1 > Access Control
7) Data Center Expert requires the ability to be able to read and write data via SNMP to the APC NMC. If only Read access is provided, Priority Scanning will not function. Ensure that there is an entry for Read and another for Write that are activated.
NOTE: Instead of two entries, one entry with the Access Type of Write+ will work for both Read (Get) and Write (Set) requests.
8) Ensure that the NMS IP/Host Name field applies to DCE. For a description of what your entry within this field means, click on Help in the upper-right corner of the NMC while on the SNMP Access page or while editing a specific SNMP Access line.
9) Take note of what the enabled Read/Write community strings are or the Write+ community string is.
10) DCE Desktop Client > Device menu > SNMP Device Communication Settings > Device Scan Settings > Search for device > Check Box on device line item > Edit Device Scan Settings.
NOTE: If selecting multiple devices, the settings that are currently displayed on the Edit Device Scan Settings screen will be set for communication between DCE and the Devices. If the devices each have different settings, this will cause lost communication to any devices that do not match these new settings that are applied.
11) The community strings recorded in step 9 should match. If they do not match, either the community strings within the Device Scan Settings or the community strings within the APC NMC SNMP Access settings must be updated.
STOP! Once the community strings are updated, navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. If the lost communication alarm does not clear after one (1) minute, proceed to step 12.
12) Increase the timeouts to 10 seconds and retries to 5. This will allow the device additional time to respond before DCE reports a lost communication with the device.
STOP! Navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. If the lost communication alarm does not clear after one (1) minute, proceed to step 13.
13) Log into the APC NMC web interface and Navigate to the Event Log. Look for any SNMP error messages, for which there are two scenarios:
- Unauthorized user attempting to access SNMP interface
- An SNMP settings is incorrect on the APC NMC or within the Device Scan Settings within DCE. Verify that the following settings are correct/matching between the systems:
- NMC SNMPv1 Access is set to enabled.
- NMC SNMPv1 Access Control settings (A&B OR C).
a) Access Type Read, NMS IP / Host Name, & Community Name
b) Access Type Write, NMS IP / Host Name, & Community Name
c) Access Type Write+, NMS IP / Host Name, & Community Name
- Device Scan Settings within DCE for the device match the community strings defined within the APC NMC.
- Device Scan Settings within DCE for the device have increased timeouts and retries.
- No SNMP Error Messages
- Traffic is blocked on the network (DCE>NMC or NMC>DCE)
- To test if traffic is getting to the NMC, edit the device scan settings within DCE and alter the community string to be incorrect. Request a device scan by right-clicking on the device within Device View. After about a minute, check the APC NMC Event log again for error messages. If there is now an error message for "Unauthorized user attempting to access SNMP Interface from XXX.XXX.XXX.XXX", SNMP traffic is able to get to the device. This would indicate that traffic may be blocked on the network from the NMC to DCE. If no error is present after this test, traffic is not getting across the network from DCE to the NMC.
14) If the tests above concluded that traffic is blocked on the network, work with your networking team to resolve the issue. Contact technical support if the issue still persists.
SNMPv3 Persistent Lost Communication Troubleshooting
1) Navigate to the Monitoring perspective and select the device group that has the device in lost communication.
2) Locate the device that is in lost communication and take note of the Application Version.
3) Right-click on the device within the Device View pane and select Launch to Device. This will load the web interface of the APC NMC.
4) Log into the web interface. Auto-login may occur if the correct credentials are defined within DCE.
NOTE: For instructions on configuring auto-login see knowledge Video: Data Center Expert | How do I configure auto-login when launching to APC devices? .
5) Navigate to the SNMP settings to determine if SNMPv3 is set to enabled.
- Application Version 6.X: Configuration menu > Network > SNMPv3 > Access > SNMPv3 Access Enable [checked]
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv3 > Access > Enable SNMPv3 access [checked]
6) Navigate to the SNMPv3 User Profile within the APC NMC.
- Application Version 6.X: Configuration menu > Network > SNMPv3 > User Profiles
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv3 > User Profiles
7) Click into the applicable profile and ensure that the fields are filled out with your specific configuration settings. Information for each of the fields, including character limitations, can be found by navigating to the help link within the upper-right of the APC NMC interface. Take note of all of the settings as they will need to be entered into DCE Device Scan Settings for the device.
6) Navigate to the SNMPv3 Access Control settings within the APC NMC.
- Application Version 6.X: Configuration menu > Network > SNMPv3 > Access Control.
- Application Version 3.X-5.X: Administration tab > Network sub-tab > SNMPv3 > Access Control.
7) Ensure that the specific user name that is used for DCE SNMPv3 communication has the Access Enabled and an applicable NMS IP/Host Name entry. To edit an entry, click on the User Name hyperlink.
NOTE: Additional information and descriptions of the fields, including what to enter within the NMS IP/Host name field can be found within the Help link in the upper-right of the APC NMC web interface.
8) DCE Desktop Client > Device menu > SNMP Device Communication Settings > Device Scan Settings > Search for device > Check Box on device line item > Edit Device Scan Settings.
9) The Authentication Passphrase, Privacy Passphrase, and protocols should match the ones recorded in step 7. Note that the Passphrase fields will be protected, so they will need to be reentered for verification. Do not copy and paste any fields as hidden characters or spaces my paste into the fields.
STOP! Navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. If the lost communication alarm does not clear after one (1) minute, proceed to step 10.
10) Log into the APC NMC web interface and change the SNMPv3 User Profile to simple settings, for testing purposes only, to:
NOTE: Instructions on navigating to these settings can be found in steps 6-7
- User Name: DataCenterExpert
- Authentication Passphrase: DataCenterExpert
- Privacy Passphrase: DataCenterExpert
- Authentication Protocol: None
- Privacy Protocol: None
11) Change the Device Scan Settings within DCE to match the new settings on the NMC, set in step 10.
DCE Desktop Client > Device > SNMP Device Communication Settings > Device Scan Settings > Search for device > Check Box on device line item > Edit Device Scan Settings.
STOP! Navigate to the DCE Desktop Client and click on the Monitoring perspective. Select the device group that contains the device in lost communication. Right-click on the device within the device-view pane and select Request Device Scan. This usually will establish communication, and is only done for testing and troubleshooting purposes. If the lost communication alarm does not clear, reach out to your local technical support for assistance. If the lost communication alarm clears, proceed to step 12.
12) Change the back to the intended settings within DCE and the NMC, ONE at a time to determine what may be causing the communication issue. Make any necessary changes to the value that is causing issues, ensuring to update it within DCE and the APC NMC.
Published for:APC Belgium
