This tool was improved and integrated into the new tool Universal Storage Collector
One more storage system I need to analyze is EMC VPLEX.
We can use VPLEX GUI to see performance data in real-time, but how to see a performance for a long period of time?
VPLEXCollector is a tool to gather performance data from VPLEX and providing it to the Graphite.
EMC VPLEX performance monitoring
EMC VPLEX uses ‘monitors’ to gather the specified statistic from the specified target at the specified interval.
Every monitor must have at least one sink, and may have multiple sinks. There are two types of sink:
- console – Sends output to VPLEX management server console.
- file – Sends output to the specified file.
Polling (collection of the specified statistics) begins when the first sink is added to a monitor.
By default, output files are located in: /var/log/VPlex/cli/reports/ on the management server. Output files are in CSV format, first row contains header, all other rows contain values for specific time.
Every monitor has a period attribute. Polling occurs automatically at the interval specified by the monitor’s period attribute. Automatic polling is disabled when period attribute is set to 0.
We can use Preconfigured monitors or Custom monitors.
Preconfigured monitors are an easy way to create monitors to collect information used to diagnose common problems.
The report create-monitors command creates three monitors for each director in the system.
Preconfigured monitors are named:
Each preconfigured monitor has one file sink. Monitors send performance data to a specified file. Output filenames are in the following format: Monitor name_Dir_nn.csv
The period attribute for preconfigured monitors is set to 0. This mean that automatic polling is disabled and we need to execute report poll-monitors command to force a poll. After that new values will appear in the output files.
To enable automatic polling we should change period attribute by executing set period command.
After that new values will appear in the output files by the new period.
It is better to use custom monitors for desired statistics from desired targets.
We can execute monitor stat-list command to see available statistics.
To start using custom monitors we should:
- Create monitor by executing monitor create command.
- Add a file sink by executing monitor add-file-sink command.
In the result we will have some files on the management server periodically updated by the VPLEX monitoring system.
VPLEXCollector is a high effective tool for gathering data from multiple VPLEXs in parallel. It connects to VPLEX Management Server by SSH and read last values from the specified monitor’s files.
Written on Scala VPLEXCollector compiled into ‘fat’ jar file (file with all dependences) and can be run on any host with Java 8 RE. It can be run in background as a daemon or as a service. It needs only one configuration file.
The source code and compiled VPLEXCollector.jar are available here: VPLEXCollector
Let’s we have directory /opt/Collector with subdirectory bin, where VPLEXCollector.jar located, subdirectory etc, where configuration file vplexcollector.xml located and subdirectory log for error log file.
To run VPLEXCollector as a service we can use this service description file vplexcollector.service:
[Unit] Description=Collecting VPLEX statistics daemon After=local-fs.target network-online.target Requires=local-fs.target network-online.target [Service] ExecStart=/usr/bin/java -cp /opt/Collector/bin/VPLEXCollector.jar vplexcollector -c /opt/Collector/etc/vplexcollector.xml Restart=on-failure [Install] WantedBy=multi-user.target
VPLEXCollector takes configuration information from special XML file:
<?xml version="1.0" encoding="UTF-8"?> <collector> <configuration> <errorlog>/opt/Collector/log/vplexcollector-error.log</errorlog> <carbon address="127.0.0.1" port="2003"></carbon> <interval>10</interval> <replacelist> <replace what=" (us)" value=""></replace> <replace what=" (KB/s)" value=""></replace> <replace what=" (counts/s)" value=""></replace> <replace what=" (counts)" value=""></replace> <replace what=" (%)" value=""></replace> </replacelist> </configuration> <vplex> <name>vplex01</name> <address>10.10.10.10</address> <username>monitor</username> <password>monitor!</password> <monitors> <monitor director="1-1-A" file="/var/log/VPlex/cli/reports/director-1-1-A.csv"></monitor> <monitor director="1-1-A" file="/var/log/VPlex/cli/reports/director-1-1-A_be.csv"></monitor> <monitor director="1-1-A" file="/var/log/VPlex/cli/reports/director-1-1-A_fe.csv"></monitor> <monitor director="1-1-A" file="/var/log/VPlex/cli/reports/director-1-1-A_fe-lu.csv"></monitor> <monitor director="1-1-A" file="/var/log/VPlex/cli/reports/director-1-1-A_storage-volume.csv"></monitor> <monitor director="1-1-A" file="/var/log/VPlex/cli/reports/director-1-1-A_virtual-volume.csv"></monitor> <monitor director="1-1-B" file="/var/log/VPlex/cli/reports/director-1-1-B.csv"></monitor> <monitor director="1-1-B" file="/var/log/VPlex/cli/reports/director-1-1-B_be.csv"></monitor> <monitor director="1-1-B" file="/var/log/VPlex/cli/reports/director-1-1-B_fe.csv"></monitor> <monitor director="1-1-B" file="/var/log/VPlex/cli/reports/director-1-1-B_fe-lu.csv"></monitor> <monitor director="1-1-B" file="/var/log/VPlex/cli/reports/director-1-1-B_storage-volume.csv"></monitor> <monitor director="1-1-B" file="/var/log/VPlex/cli/reports/director-1-1-B_virtual-volume.csv"></monitor> </monitors> </vplex> </collector>
This file describes:
- Internal configuration
- Description of VPLEXs
Here we can see:
- errorlog – path to the file for logging errors
- carbon – address and port of the Carbon server
- interval – time in min between data gathering
- replacelist – ordered list of replacements in output from VPLEX before output to Carbon
Description of VPLEXs
Each VPLEX should specified in a separate section .
Here we can see:
- name – name of VPLEX to specify it in the Carbon address – Management Server address
- username – username for ssh
- password – password for ssh
- monitor – monitor’s file name and director it attached to
Graphite and Grafana
For each VPLEX we can create one or several dashboards in Graphite and/or Grafana.