[an error occurred while processing this directive]

ViPR SRM 3.6 – ViPR SRM Troubleshooting

Table of Contents

Overview

Locate specific ViPR SRM troubleshooting procedures, receive general recommendations for customer troubleshooting, and learn how to collect diagnostics.

Back to Top

General recommendations for customer troubleshooting

Unless Customer Service asks you to perform different actions, follow these recommendations:

Back to Top

SSL certificates

Generally, EMC products are distributed with a self-signed SSL certificate. If you receive an SSL certificate error or a blank pane when you are attempting to access content served by a data source, resolve the issue by doing one of the following procedures.

SSL certificate error examples provides a list of sample errors.

Temporary resolution

Accept the certificate exception temporarily: Troubleshooting an SSL certificate error: Accepting the exception.

Permanent resolution

Add an SSL certificate to your truststore to permanently prevent the exceptions from appearing.

You can: Perform:
Install the self-signed SSL certificate that is provided with the EMC product using an Internet Explorer browser. Installing a self-signed SSL certificate from an Internet Explorer browser
Install the self-signed SSL certificate that is provided with the EMC product using a Chrome or Firefox browser.
  1. Export the self-signed SSL certificate to save it to the export location: Exporting an SSL certificate from an Internet Explorer browser
  2. Importing an SSL certificate into your truststore
Import an exported SSL certificate that you or another administrator prepared. Importing an SSL certificate into your truststore

If you need to prepare an SSL certificate to share with other users, use: Exporting an SSL certificate from an Internet Explorer browser.

Back to Top

SSL certificate error examples

The SSL certificate error or a blank pane appears because the certificate is self-signed and it is not added as an exception or into the truststore yet. For example, if you are attempting to access content served by a data source, a blank pane appears when you select views under SolutionPacks.

Chrome error examples

This site's security certificate is not trusted!

Or, a blank pane.

Firefox error example

This Connection is Untrusted

Internet Explorer error examples

There is a problem with this website's security certificate.
Content was blocked because it was not signed by a valid security certificate.

Back to Top

Troubleshooting an SSL certificate error: Accepting the exception

An SSL certificate error appears in your browser.

To resolve this issue,

Procedure

Accept the exception.
Steps may vary for your browser. For example, for Internet Explorer, click Continue to this website (not recommended).
This temporary solution causes the exception to be ignored for the current browser session, but the exception reappears if you open another browser tab or window.
Back to Top

Installing a self-signed SSL certificate from an Internet Explorer browser

Install the self-signed SSL certificate that is provided with the EMC product to prevent certificate exception errors.

The following procedure is for Internet Explorer version 9.0. The steps may vary slightly depending on your browser version.

Procedure

  1. Click Continue to this website (not recommended) in the browser window.
  2. Click the red Certificate Error indicator next to the address bar.
  3. In the Untrusted Certificate window, select View certificates.
  4. In the Certificate dialog box, in the General tab, click Install Certificate.
  5. Follow the instructions in the Certificate Import Wizard.
    1. Select the Place all certificates in the following store setting to add the certificate to your truststore.
  6. In the Certificate dialog box, in the General tab, click OK.

After you finish

Refresh the browser window.

Back to Top

Importing an SSL certificate into your truststore

Import an SSL certificate into your truststore to prevent certificate exception errors.

Procedure

Back to Top

Importing an SSL certificate in Chrome

For Chrome, use this procedure to import the certificate into your truststore.

The following procedure is for Chrome version 28.0.1500.95. The steps may vary slightly depending on your browser version.

Procedure

  1. Go to Settings > Show advanced settings.
  2. Scroll down and select HTTP/SSL > Manage Certificates.
  3. In the Trusted Root Certification Authorities tab, click Import.
  4. Follow the instructions in the Certificate Import Wizard to import the certificate.
  5. Click OK in Internet options.
  6. Refresh the browser window.
Back to Top

Importing an SSL certificate in Firefox

For Firefox, use this procedure to import the certificate into your truststore.

The following procedure is for Firefox version 23.0. The steps may vary slightly depending on your browser version.

Procedure

  1. Go to Firefox > Options > Options.
  2. Click Advanced.
  3. In the Certificates tab, click View Certificates.
  4. Click Import.
  5. Locate the certificate file and click Open.
  6. Click Save.
  7. Click OK in Certificate Manager.
  8. Refresh the browser window.
Back to Top

Importing an SSL certificate in Internet Explorer

For Internet Explorer, use this procedure to import the certificate into your truststore.

The following procedure is for Internet Explorer version 9.0. The steps may vary slightly depending on your browser version.

Procedure

  1. Go to Tools > Internet options.
  2. In the Content tab, click Certificates.
  3. Select the Trusted Root Certification Authorities tab.
  4. Click Import.
  5. Follow the instructions in the Certificate Import Wizard to import the certificate.
  6. Click OK in Internet options.
  7. Refresh the browser window.
Back to Top

Exporting an SSL certificate from an Internet Explorer browser

Use this procedure to export a self-signed SSL certificate, so that you can either import the resulting exported certificate into your truststore or share it with other users.

The following procedure is for Internet Explorer version 10.0. The steps may vary slightly depending on your browser version.

Procedure

  1. Access the host for which you want to export the SSL certificate from the browser.
    For example: https://<myHostname>.emc.com
  2. Select Continue to this website (not recommended) in the There is a problem with this website's security certificate window.
  3. Click the red Certificate Error indicator next to the address bar.
  4. In the Untrusted Certificate window, select View certificates.
  5. In the Certificate dialog box, select the Details tab and ensure that the Show field is set to All. Click Copy to File.
  6. In the Certificate Export Wizard,
    1. Click Next in the Welcome screen.
    2. Click Next to accept the default DER encoded binary X.509 setting in the Export File Format screen.
    3. Browse to the location where you want to save the certificate.
    4. Specify the certificate file name and click Save.
    5. In the File name field, ensure that the location and the certificate file name are correct. Click Next.
    6. Click Finish.
    7. Click OK when The export was successful appears.
  7. In the Certificate dialog box, click OK in the Details tab.
  8. Share the resulting exported SSL certificate with other users to use.
Back to Top

Switch discovery failure in the SolutionPack for Cisco MDS/Nexus

Edit and exclude the GENERIC-INTERFACES capability on Cisco MDS/Nexus switches.

Discovery must be successful for every switch that the SolutionPack for Cisco MDS/Nexus displays. The value for the devtype must be FabricSwitch. For most switches, this information is discovered automatically. However, if your switch discovery does not occur with devtype equal to FabricSwitch, the switch will not appear under the SolutionPack reports.

Procedure

  1. Open the Device Discovery GUI ( http://yourserver_ip:58080/device-discovery/).
  2. Edit SNMP agent settings (under Devices), and exclude the GENERIC-INTERFACES capability on Cisco MDS/Nexus switches.

    Without the capability, more polling occurs.

    Device discovery GUI

    Edit the Device form

Back to Top

Collecting diagnostics on UNIX

You can run the diagnostic.sh script to collect data for Customer Support. The script collects files required for troubleshooting EMC M&R and compresses them into a zip file.

Procedure

  1. Log in to a UNIX machine with root access.
  2. Access the diagnostic.sh script at /opt/APG/Tools/APG-Diagnostic-Tools/Default/bin/diagnostic.sh and run it.
    If EMC M&R is installed as Minimal, the script is missing from the installation and you need to run # manage-modules.sh install diagnostic-tools to install the script.
    Output from the script goes into a zip file in the /tmp/ directory. The output file name is APG-DiagnosticFiles.tar.gz
  3. Upload the output file into a Service Request or if the file is large, to an FTP site specified by Customer Support.
Back to Top

Collecting diagnostics on Windows

You can run the diagnostic.sh script to collect data for Customer Support. The script collects files required for troubleshooting EMC M&R and compresses them into a zip file.

Procedure

  1. Log into a Windows machine and open a command window.
  2. Navigate to the folder <%INSTALL_ROOT%>\Program Files\APG\bin.
    If EMC M&R is installed as Minimal, the diagnostic.sh script it missing and you need to run manage-modules.cmd install diagnostic-tools to install the script.
  3. Run diagnostic.cmd.
    Output goes into the file C:/Program Files/APG"\APG-DiagnosticFiles.zip.
  4. Upload the file into a Service Request or if the file is large, to an FTP site specified by Customer Support.
Back to Top

Virus scanner problems with SolutionPack for Microsoft SQL Server

Take action to prevent a virus scanner such as Norton/Symantec Anti-Virus from causing performance problems or misidentifying MySQL files as containing spam.

Procedure

  1. After installing MySQL Server, disable virus scanning on the main directory (datadir) used to store your MySQL table data.
    Virus-scanning software usually has an option that enables specific directories to be ignored.
  2. To prevent the temporary files also being scanned, configure a separate temporary directory for MySQL temporary files and add this directory to the virus scanning exclusion list. To do this, add a configuration option for the tmpdir parameter to your my.ini configuration file. For assistance, go to http://dev.mysql.com/doc/refman
Back to Top

Detect and repair database crashes

Procedure

  1. Perform a weekly check:
    1. Browse log files.
    2. Check for suspicious errors.
  2. If you find slow reports, data missing, a portal hanging, and so on, suspect a database crash and closely check the log files. For instance, if the database crashes from an import-properties" task, you may receive errors like this:
    140611 13:16:23 : INFO: Importing data from 'data_property' into '/opt/APG/Databases/APG-Property-Store/Default/tmp/DB/APG-DB'...
    140611 13:16:23 : Jun 11, 2014 1:16:23 PM com.watch4net.propstore.apg.ErrorLoggingCallback onTaskFailed
    140611 13:16:23 : SEVERE: Importing data from 'data_property' into '/opt/APG/Databases/APG-Property-Store/Default/tmp/DB/APG-DB' failed !
    140611 13:16:23 : java.sql.SQLException: Table 'data_property' is marked as crashed and should be repaired
    .
    .
    .
    140611 13:16:23 : Jun 11, 2014 1:16:23 PM com.watch4net.propstore.apg.ErrorLoggingCallback onTaskFailed
    

    MySQSL errors could appear:

    140611 13:16:23 [ERROR] /opt/APG/Databases/MySQL/Default/bin/mysqld: Table 'data_property' is marked as crashed and should be repaired
    140611 13:16:23 [ERROR] /opt/APG/Databases/MySQL/Default/bin/mysqld: Sort aborted: Table 'data_property' is marked as crashed and should be repaired
    
  3. If a database crash occurs, you should run mysql-database-check.sh:
    [root@lglbb142 APG]# ./bin/mysql-database-check.sh
    Enter username: [apg]
    
    Enter database: [apg]
    
    Enter password:
    # Connecting to 127.0.0.1...
    apg.cache_group_0_1401278400                       OK
    [...]
    apg.cache_group_86400_1399680000                   OK
    apg.data_property
    error    : Size of datafile is: 2         Should be: 3758440
    error    : Corrupt
    apg.data_property_flat                             Table is already up to date
    apg.data_variable                                  OK
    
    Repairing tables
    apg.data_property
    Error    : Incorrect information in file: './apg/data_property.frm'
    error    : Corrupt
    # Disconnecting from 127.0.0.1...
  4. Depending on the severity of the crash, you may or may not be able to restore the table.
    You may need to restart affected services.
Back to Top

Leftover components can cause VNX installation problems

After removing all listed VNX instances from the SolutionPacks view for VNX, leftover components from a removed instance can remain and can cause problems in reinstalling the SolutionPack. You can find these components by searching in the Logical Overview or in its Collecting subsection. Manually uninstall these components.

Back to Top

Find the SolutionPack that discovered a device

Procedure

  1. Click the down arrow next to the quick search field.
    A popup screen appears.
  2. Click the Filter field to display a list of options and select Edit expression.
  3. Type an expression. For example:
    device=='000195700930'
  4. Click OK.
  5. in the Expansion field, replace “device” by "device, source".
  6. In the Search Base field click the down arrow and select search from root.
  7. Click Apply.
    form fields

    Finding the collector for a device

    All devices that meet the search criteria and all collectors associated with them are displayed. The name of the collector enables you to identify the SolutionPack associated with the collector.

    You can filter the column using the filter icon displayed in the table title.

Back to Top

Troubleshooting topics in SolutionPack articles

Many SolutionPack articles contain troubleshooting topics that apply to specific SolutionPacks.

Back to Top

Known issue with user-defined roles

If an SNMPv3 user or SNMPv1/v2c community string used for switch discovery is assigned to a role that has VSAN policy set to deny, there is a possibility of switch reboot and supervisor fail-over.

This is due to an issue in Cisco NX-OS 5.2(6) through 6.2(x).

The workaround is to execute the following commands in sequence:

linbge197#config t
linbge197#role name <name of the role>
linbge197#no vsan policy deny
linbge197#copy run start
linbge197#exit

Back to Top

Troubleshooting alert collection errors on SolutionPack for EMC Data Protection Advisor

Reconfigure the data collection options in the SolutionPack for EMC Data Protection Advisor if alert collection is not occurring. This procedure applies to each DPA instance for which alert collection is not occurring.

Procedure

  1. Click Administration.
  2. Click Centralized Management.
  3. Click Storage > EMC Data Protection Advisor .
  4. Within the Instance column, click on the edit icon to reconfigure the options for the affected DPAinstance that is affected by the problem.
  5. Select the Enable alert collection from DPA.
  6. Click Reconfigure.
Back to Top

Troubleshooting report generation errors on SolutionPack for EMC Data Protection Advisor

Reconfigure the data collection options in the SolutionPack for EMC Data Protection Advisor if reports are not being generated. This procedure applies to each DPA instance for which reports are not being generated.

Procedure

  1. Click Administration.
  2. Click Centralized Management.
  3. Click Storage > EMC Data Protection Advisor .
  4. Within the Instance column, click on the edit icon to reconfigure options for the affected DPA instance.
  5. Select the Do you want to configure advanced settings and Enable scheduling of DPA reports options.
  6. Click Reconfigure.
Back to Top

Limitation: MDisks column is blank in volume reports

MDisks column for mirror volumes is blank in volume reports.

There is no workaround.

Back to Top

Limitations

Discovery of LDAP users requires active GUI session

Due to limitations in the IBM software, discovery using the LDAP users account will not succeed unless there is an active IBM XIV GUI user session. The session must be active either for the corresponding LDAP user or for a storage administrator.

Max Storage Volume Response Time metric not collected

The SolutionPack does not collect the Max Storage Volume Response Time (ms) metric needed for the All > Explore > Hosts > [host] > Device Summary > [host] report.

No support for Used for File, LUN alias, and Replica 2 and Replica 3 details

‘Used for File’ metrics are not supported. Therefore, the respective columns in the following reports will be empty.

  • All > Dashboards > Storage > Enterprise Capacity Dashboard > Usable Capacity by Pool
  • All > Dashboards > Storage > Chargeback by Group
  • All > Explore > Storage > More Reports > Storage Pools

‘LUN Alias’ column is empty in the following report:

  • All > Explore > Storage > More Reports > More Storage Inventories > All LUNs

Remote replication data is not exposed through the SMI provider; therefore, Replica 2 and Replica 3 columns details will be empty in the following report:

  • All > Explore > Storage > More Reports > More Storage Inventories > Replication

Back to Top

Supported use cases and limitations

Limitations in global reports

Used capacity is the same as the total capacity for Ontap 7.x versions of the following reports:

  • Explore > Storage > More Reports > Service Level Capacity
  • Explore > Storage > More Reports > More Storage Inventories > Array by Service Level

In Dashboards > Storage > Enterprise Capacity Dashboard > Usable Capacity > Used for Block, the OnTap 7.x CLI does not provide LUN used capacity therefore the total capacity is used for computation.

The following global reports/metrics are not applicable:

  • In Explore > Storage > More Reports > More Storage Inventories > NAS File Systems, Snapshot Total, Snapshot Used, and Snapshot Utilization are not applicable
  • In Explore > Storage > More Reports > Storage Pools, Overhead Capacity is not applicable in the File Systems/Flex Vols report and Pool Enabled and Pool Overhead are not applicable in the Subscriptions report
  • Explore > Storage > More Reports > RAID/Parity Group
  • Explore > Storage > More Reports > NAS Pools

In Report Library > NetApp Filer > Storage Systems Summary > 7-Mode > Filer CPU Utilization, the "Uptime" and "System CPU(%)" columns will be empty for OnTap 8.1 or higher versions that are configured as 7-Mode.

In Dashboards > Storage > Enterprise Capacity Dashboard > Used Capacity by Type and Explore > Storage > More Reports > Enterprise Capacity, the "Used for HDFS" and "Used for Object" metrics are blank for NetApp.

In Explore > Storage > More Reports > More Storage Inventories > NAS File Systems, for filers running OnTap 8.2, the total capacity of a thin file system will be the same as its used capacity.

Back to Top

Troubleshooting

Report display problems

Back to Top

What to do if data does not appear in reports

Procedure

  1. Do the following to check the Collector Manager logs:
    1. View /opt/APG/Collecting/Collector-Manager/microsoft-sqlserver/logs/collecting-0-0.log.
    2. Verify that the last message in the log states that a certain number of raw values were collected using 13 SQL queries. This message indicates the collector is working.
      For example: <instance_master> collect done: 2,735 new raw values collected in 0 seconds 527 ms using 13 SQL queries
    3. Check for any Java exceptions and warning messages.
  2. If collector errors exist, do the following:
    1. Ping the Microsoft SQL Server using the fully qualified domain name (FQDN).
    2. Replace the current user, which is the username and password set during SolutionPack installation to retrieve data from the Microsoft SQL sever, with a user with SYSADMIN rights.
    3. Use telnet to check if the Microsoft SQL port is accessible from the SQL collector server.
    4. Reconfigure the SolutionPack by navigating to Centralized Management > SolutionPacks > Microsoft SQL Server.
    5. Under SolutionPackBlock, click the pencil next to Collector to reconfigure the settings.
    6. To edit advanced connection settings, you may need to manually edit the SQL collector configuration file.
  3. If no errors exist and the last message in the log indicates that 13 queries were executed, do the following:
    1. Wait a minimum of three polling cycles. By default, one polling cycle is five minutes.
    2. Navigate to Centralized Management > Scheduled Tasks.
    3. Under the Backend category, run backend/Default/import_variable_status.task.
    4. Refresh the reports after five minutes.
  4. Make sure the Frontend is attached to the database used by the SQL collector.
Back to Top

Running a scheduled task to import data into reports

After you push a new configuration into a collector, a scheduled task runs and populates the reports with new data. You can manually run the scheduled task to import the data more quickly.

Before you begin

Allow at least three polling cycles to pass before manually running the scheduled task.

Procedure

  1. Click Administration.
  2. Click Centralized Management.
  3. Expand Scheduled Tasks.
  4. Click Database.
  5. Select the import-properties-Default task.
  6. Click Run Now.
  7. Confirm success in running the task in the Last Result and Last Result Time columns.
Back to Top

Viewing collector errors in the Collector-Manager log files

Review the Collector-Manager log files to troubleshoot problems with data collection.

Procedure

  1. Click Administration.
  2. Click Centralized Management.
  3. Expand Collecting.
  4. Click the Collector-Manager for your collector instance.
    Collector-Manager::<Collector-Manager instance> - <physical_host_ID>
  5. Expand Log Files and click the View File icon to review the configuration error messages.
Back to Top

Limitation: some alerts not displayed

ViPR SRM fails to receive and display some alerts

If a trap is lost in the network communication, Solutions Enabler is not aware of the host and is not able to resend the lost information.

Back to Top

Troubleshooting

Report display problems

Back to Top

What to do if data does not appear in any reports

Procedure

  1. After the completion of at least three collection cycles, verify if data is populating into the reports. If there is still no data in the reports, continue to the next step.
  2. Run the scheduled task to import data into reports. If there is still no data in the reports, continue to the next step.
  3. To view the log files for errors, go to Centralized Management and click Collecting > Collector-Manager::<instance name> > Log Files.
Back to Top

Running a scheduled task to import data into reports

After you push a new configuration into a collector, a scheduled task runs and populates the reports with new data. You can manually run the scheduled task to import the data more quickly.

Before you begin

Allow at least three polling cycles to pass before manually running the scheduled task.

Procedure
  1. Click Administration.
  2. Click Centralized Management.
  3. Expand Scheduled Tasks.
  4. Click Database.
  5. Select the import-properties-Default task.
  6. Click Run Now.
  7. Confirm success in running the task in the Last Result and Last Result Time columns.
Back to Top

What to do if data does not appear in some reports

Procedure

  1. Run the scheduled task to import data into reports. If there is still no data in the reports, continue to step 2.
  2. Search for the metric in the database.
  3. To view the log files for errors, go to Centralized Management and click Collecting > Collector-Manager::<instance name> > Log Files.
Back to Top

Searching for metrics in the database

You can verify that a metric is being collected and used for reporting when you search and find the metric in the database.

Procedure
  1. Go to the Administration page.
  2. Under Modules, click Management of Database Modules.
  3. On the Metric Selection page, create the filter, type the number of results, and select the properties to display for the metric.
    For example, to list up to 100 results of the SignalWait Time metric with the properties of device and IP, type name=='SignalWaitTime' in the Filter field, 100 in the Maximum results field, and select device and IP for the Properties to show.
  4. Click Query.
    A list of the metric results appears. If nothing displays, the metric is not being collected.
Back to Top

Viewing collector errors in the Collector-Manager log files

Review the Collector-Manager log files to troubleshoot problems with data collection.

Procedure

  1. Click Administration.
  2. Click Centralized Management.
  3. Expand Collecting.
  4. Click the Collector-Manager for your collector instance.
    Collector-Manager::<Collector-Manager instance> - <physical_host_ID>
  5. Expand Log Files and click the View File icon to review the configuration error messages.
Back to Top
[an error occurred while processing this directive]