ELK Stack
more flexibility to do great things!
About ELK Stack
Elasticsearch, Logstash, and Kibana, when used together is known as an ELK stack.
The ELK Stack is a collection of three open-source products — Elasticsearch,Logstash, and Kibana — all developed, managed and maintained by Elastic. Elasticsearch is a NoSQL database that is based on the Lucene search engine. ... Logstash collects and parses logs, and then Elasticsearch indexes and stores the information.
- Real-time data and real-time analytics
- Scalable, high-availability, multi-tenant
- Accessibility of Log Data
ELK Stack Topics
The following are the things covered under ELK.
Data Pipeline
Build your own data pipeline using the ELK stack
Different types of Plugins
Familiarize yourself with the key features of Logstash and the variety of input, filter, and output plugins it provides
Visualizations
Create actionable insights using charts, histograms, and quick search features in Kibana4
log analysis
Understand the need for log analytics and the current challenges in log analysis
Custom Plugins
Build your own custom Logstash plugin
Role of ElasticSearch
Understand the role of Elasticsearch in the ELK stack
Course Contents
The following are the course contents offered for ELK Stack
- The need for log analysis
- Issue debugging
- Performance analysis
- Security analysis
- Predictive analysis
- Internet of things and logging
- Non-consistent log format
- Tomcat logs
- Apache access logs – combined log format
- IIS logs
- Variety of time formats
- Decentralized logs
- Expert knowledge requirement
- Elasticsearch
- Logstash
- Kibana
- ELK data pipeline
- ELK Stack installation
- Installing Elasticsearch
- Running Elasticsearch
- Elasticsearch configuration
- Network Address
- Paths
- The cluster name
- The node name
- Elasticsearch plugins
- Installing Logstash
- Running Logstash
- Logstash with file input
- Logstash with Elasticsearch output
- Configuring Logstash
- Installing Logstash forwarder
- Logstash plugins
- Input plugin
- Filters plugin
- Output plugin
- Installing Kibana
- Configuring Kibana
- Running Kibana
- Kibana interface
- Discover
- Visualize
- Dashboard
- Settings
- Input dataset
- Data format for input dataset
- Configuring Logstash input
- Filtering and processing input
- Putting data to Elasticsearch
- Running Kibana
- Kibana visualizations
- Building a line chart
- Building a bar chart
- Building a Metric
- Building a data table
- Configuring Logstash
- Logstash plugins
- Listing all plugins in Logstash
- Data types for plugin properties
- Array
- Boolean
- Codec
- Hash
- String
- Comments
- Field references
- Logstash conditionals
- Input plugins
- Output plugins
- Filter plugins
- Codec plugins
- Logstash plugin management
- Plugin lifecycle management
- Installing a plugin
- Updating a plugin
- Uninstalling a plugin
- Structure of a Logstash plugin
- Required dependencies
- Class declaration
- Configuration name
- Configuration options setting
- Plugin methods
- Input plugin
- Filter plugin
- Output plugin
- Codec plugin
- Writing a Logstash filter plugin
- Building the plugin
- Why Elasticsearch?
- Elasticsearch basic concepts
- Exploring the Elasticsearch API
- Listing all available indices
- Listing all nodes in a cluster
- Checking the health of the cluster
- Health status of the cluster
- Creating an index
- Retrieving the document
- Deleting documents
- Deleting an index
- Elasticsearch Query DSL
- Elasticsearch plugins
- Kibana 4 features
- Kibana interface
- Discover page
- Querying and searching data
- Field searches
- Range searches
- Special characters escaping
- New search
- Saving the search
- Loading a search
- Field searches using field list
- Visualize page
- Creating a visualization
- Visualization types
- Visualizations
- Area chart
- Data table
- Line chart
- Markdown widget
- Metric
- Pie chart
- Tile map
- Vertical bar chart
- Dashboard page
- Building a new dashboard
- Saving and loading a dashboard
- Sharing a dashboard
- Input dataset
- Configuring Logstash input
- Grok pattern for access logs
- Visualizing with Kibana
- Running Kibana
- Searching on the Discover page
- Visualizations – charts
- Building a Line chart
- Building an Area chart
- Building a Bar chart
- Building a Markdown
- Dashboard page
- Prevention of data loss
- Data protection
- System scalability
- Data retention
- ELK Stack implementations
- ELK at SCA
- ELK Stack at LinkedIn
- ELK at Cliffhanger Solutions
- Kibana demo – Packetbeat dashboard
- Elasticsearch plugins and utilities
- Curator for index management
- Curator commands
- Curator installation
- Shield for security
- Shield installation
- Adding users and roles
- Using Kibana4 on shield protected Elasticsearch
- Marvel to monitor
- Marvel installation
- Marvel dashboards
- ELK roadmap
- Elasticsearch roadmap
- Logstash roadmap
- Event persistence capability
- End-to-end message acknowledgement
- Logstash monitoring and management API
- Kibana roadmap
Download
Download ELK Stack course plan