Icinga2 Advanced Training is designed especially for those who want to know all the savvy hooks of such a swiss knife of monitoring as icinga2. The absolute minimum required information on the open-source monitoring solution one can obtain on Icinga Fundamentals training. In this course, we teach a comprehensive configuration of the system. Besides, it is full of useful shortcuts, automation tips, and best practices that will save your time for inventing yet another wheel.
icinga2 is a Swiss knife of monitoring
— icinga deployment engineer in a philosophy mood
We in Dexor are in love with monitoring (of everything), and starting from 2018, we are partnering with the Icinga team. So we have expertise in designing, setting up, and maintaining the icinga2 monitoring solution. Moreover, we are providing different training courses for you.
In contrast to the current Icinga2 Fundamentals Training, the Advanced training focuses mainly on configuring the monitoring system by modifying the configuration files. Although this is not the first choice if you are considering rolling out the monitoring system (best practices recommend choosing Icinga Director), this approach makes it easier to understand how the system works in detail, and we find it rather good.
Icinga2 Advanced training. Summary
Cluster
Distributed monitoring is one of the most crucial features of icinga2. We will discuss for each scenario whether it is an absolute must to use it. We will dive into SSL terminology because it protects all internal icinga2 connections within a cluster. Before creating icinga2 zones, we will learn about network segment design and communication within zones. We will define a “cluster” in icinga2 by creating Icinga objects in the monitoring system’s configuration files.
One of the main features of the cluster in icinga2 is that the configuration only needs to be done in one place and appears on the connected cluster hosts through Icinga. Keeping this in mind, we have everything we need to configure HA for Icinga master nodes and Master – Sattelite relationships between connected hosts.
Advanced configuration
Icinga2 is a very flexible tool. One of the key reasons for it is its DSL – Domain-specific language. We will learn advanced apply rules based on arrays and dictionaries. We will find out which embedded functions have icinga2, and we will see how we can use them in the object configuration for accessing object attributes. Finally, we will define our custom function for our objects in the icigna2 configuration files.
Icinga2 API
Icinga2 API is a significant feature of icinga2 that extends internal API with HTTP requests, so it is possible to perform actions, create/modify config objects, and many more. We will learn how to equip API users with sufficient permissions and the necessary credentials for acting in the way you like. We will see how the API user may access icinga2 config files, run actions, subscribe on event streams, and query statuses of icinga2 objects. Finally, we will see how GUI – icingaweb2 employs the same API for some particular actions – so we will discover the internal logic of icingaweb2 for us a little.
Troubleshooting
Icing 2 is flexible and complex enough software. It is its strength and weakness at the same time. Unfortunately, there is no magic button “Make everything work. Please!”. But there are some general places to dig deeper to find the source of the error. We will discuss the most common sources of errors, and find out which methods are available for us. For example, sometimes, it is enough to make some actions manually. Or maybe, it is necessary to enable a debug log of icinga2 to find the error source. We will also cover possible notification problems, and sources of eventual cluster errors.
Unfortunately, there is no magic button “Make everything work. Please!”
— Icinga support engineer in a philosophy mood
Director Module
Director is one of the most recommended modules of Icinga Web 2, which is considered as a best practice to create and modify icinga2 objects, instead of modifying configuration files.
On the example of the Icinga director, we will see how we, in general, install modules for Icinga Web2. We will configure the module for interaction with icinga2 core, prepare it for the correct work and create some templates, which we will use further. The detailed explanation of how to work with the director you may get on our Icinga Fundamentals Training.
Nevertheless, in Icinga2 Advanced training, we will discover one of the most significant features of director – automation. On the example of MySQL database, we will see how to configure automation jobs, which may read DB, interpret fields in the way you like, and create icinga2 objects according to your needs. As an import source Icinga Director can use different information bases, for example, connected AWS account, Azure, database, AD/LDAP, and even CSV files.
Icinga can regularly import the data from selected sources so that all created hosts registered in your system appearing in the monitoring system without your assistance.
Java Monitoring
We will explore how to expose Java’s standard for the management of applications – JMX locally or remote via JSR-160. On the example of a java web application with Jolokia, which uses JMX via JSON interface, we will pass local and remote JMX requests. For this, we will employ the JMX4Perl library, which connects to Jolokia. On the example of the check_jmx4perl plugin, we will fire some queries for available MBeans and use them in icinga2 services for java application monitoring.
Log Monitoring
Logs are helping us to find errors and critical states of our applications. As soon as we get an error, we dive into the log files and try to find something that will tell us what happened in the system. The ability to have the sources of errors at any given moment gives us monitoring of the key-log files. In Icinga2 Advanced training, we will learn two approaches to working with log files – a local and a centralized one.
For the local approach, we will employ two plugins – check_log, and check_logfiles. For the centralized method, we can work with Elasticseach Logstash Kibana Stack, or a Graylog – an alternative to ELK. We use Logstash in the training course.
VMware Monitoring
For monitoring of the different aspects of VMware virtual environments, we will discover several plugins based on vSphare SDK for Perl, which is available for free after registration on VMware.
We will discuss the following plugins:
- check_vmware_esx – for general-purpose monitoring
- check_vmware_snapshots – for snapshots monitoring
- check_esxi_hardware – monitoring of the host system health
Graphing
On the example of Graphite, we will see how to visualize the performance data of the checks. For this, we firstly configure a Graphite scheme for storing the performance data of icinga2. Enable and configure graphite module for Icinga Web 2. We will find out how Graphite stores the data, how we can set thresholds for their visualization in the graphs, and how to monitor the graphing process. Moreover, we will see how to define visualization of custom, not predefined check.
After Icinga2 Advanced training
The engineer who has completed the training will be able to:
- troubleshoot complex faults of the installed monitoring system;
- design and implement a sophisticated cluster of the monitoring system with sufficient reasoning of his choice;
- administrate and maintain the icinga2 setup;
- guide and encourage colleagues to use monitoring tools in everyday business.
- make a precise tuning of the permissions of icinga2 and Icinga Web2 for the required API users;
- set up the automatic integration of hosts into the monitoring system according to the requirements;
- create challenging services with, for example, variable thresholds.
