Tag: Splunk

Few important tips to take note of

Here we can see about Field Extraction and table formatting

| fields – to make searches more efficient, main part of search

• -fieldname fieldname2 is different for – fieldname fieldname2

it makes search more efficient if the fields are included or excluded.

| table – like fields- but changes data to tabulated format (transforming command)

fields to be used before using the table command

| dedup – to remove duplicate values form the values displayed in the events/rows

1. count ex:
2. top, limit: top vendor limit=5
3. rare
4. showperc=true/false
5. otheruser=true

Few of the Stats commands

1. count ex: stats count as “column name” by field name
2. distinct count ex:
3. sum
4. average
5. min
6. max
7. list
8. values

Chart

| chart count over status

count of a field can be noted in x axis, y always numeric as the data mentioned in x axis can be displayed based on the count.

Questions and Answers:

1. Which of the following removes the duplicate? dedup.
2. In a single series data table, which column provides the x-axis values for the visualization? 1st column?
3. Which optional argument of the addtotals command changes the label for row totals in a table? label.
4. Which clause can be used with the top command to change the name of the count column? countfield.
5. Which clause can be used with the top command to specify a number of values to return? limit.
6. Which command changes the appearance of field values? fieldformat.
7. How many columns are displayed by default when using the chart command? 10.
8. Which type of default map visualization uses shading to represent relative metrics? chloropleth.
9. Which of the following commands can return a count of all events matching a search criteria over a specific time period? trendline sma/ema/wma
10. When using the time chart command, which axis represents time? x-axis.
11. How can the order of columns in a table be changed? By changing the order of fields specified in the table command.
12. Which argument can be used with the time chart command to specify the time range to use when grouping events? span.
13. Which clause can be used with the rare command to specify whether a percentage column is created? showperc.
14. Which command can be used to exclude fields from search results? fields.
15. Which argument can be used with geostats command to control the column count? globallimit.

Business Analyst – Splunk related project

• Excellent troubleshooting and problem-solving skills to identify problems from a functional perspective, specifically when supporting end-user testing and training. Includes proficiency with such tools as Splunk, AppDynamics, etc.
• Create reports, dashboards, and visualizations to understand business performance
• Analyze process issues and bottlenecks and to make improvements
• Communicate and validate requirements with relevant stakeholders
• Develop and maintain reporting tools
• Perform data discovery, analysis, and modeling
• Collaborate with product manager on roadmap planning and prioritization

Splunk introduction – notes!

Splunk is considered Google like search engine for the logs. Correlation of data is one of the key features considered to use Splunk.

Flexible data pipeline – any type of data can be roped into the platform, extract, and format it and make it searchable

Quick search, time normalization and powerful query language makes it stand top across competitors

ADHOC Search- considered in general inefficient on comparison with other types of searches.
As u are trying to find the problem- may-may not find it after the search – if many people do at the same time, efficiency may be impacted. It is done to make a feel for the data, to pin down the issue we are looking for! the discovery of issues which are already known and few which the user sees for the first time as well.

Scheduled Search-you know the problem- search in time intervals and make it efficient. During this time, we must make sure- the impact on the system is high| real time Search- real time search as it happens – heavy impact on environment- do not perform any real time without approval.

Licensing model – earlier- charge on amount of data bringing in- usually- filter the data – so correlating gets impacted-

Workload pricing model- computation charges- based on the compute on platform the charge is made and not for the data loaded-in. more computation, the licensing will be decided.

1. Major features of Splunk enterprise.
2. index – bucket of data -> as data enters its inspected and match to a source type and make it as a single event – timestamped and stored in the Splunk indexes so it can be searched. a particular level of access ex: network logs to a index, application logs to another index etc.
3. index can be considered for the data retention ex: 30 days, 60 days
4. by searching in the Splunk- diff source type can be searched
5. Search – monitor- alert
6. one can create alerts and monitor specific conditions
7. allows you to collect reports in visualization and dashboard

Web Interface

1. Apps – sit on top of Splunk instance, can also be called as workspace
2. Roles- decide what the user can see, do or interact with
   1. Administrator: role is the powerful role in the list of roles; – install app- ingest data- create knowledge objects for all users
   2. Power User: create and share knowledge objects for users of an app and do real time searches. – this is in general people get to create alert and dashboards.
   3. User role – can see only their own knowledge objects and those shared with them.

Once logging in Splunk enterprise, it has 2 apps by default- and there are so many apps which can be picked from the Splunk base!

1. Home app – manage other apps- gives quick space to – create custom dashboard as a default
   Admin can also add apps from home app
2. Search & Reporting app: provides a default interphase for searching and analyzing the data and has 8 components
   1. Splunk bar-> edit->view messages->monitor the progress of search jobs
   2. App bar
   3. Search bar- used to run searches
   4. time range picker – events for specific time ex: 60 min, 1 day, 4 day – – do not perform long time search
   5. histogram – the events occurring in the specific period is not here.
   6. how to search panel
   7. Data Summary button
      1. host (IP address, domain name)
      2. source(path/filename)
      3. source type (classification of data)
3. Table view-
4. Search history- old search history can be searched with the filter option, can be re-run again across specific timeline on how many runs have been made.
5. rolling over events- makes it highlighted- can add that to search
6. failed password to the search – can remove the data from search by clicking on the highlight
7. drop down for event actions.
8. The vents can be extracted by clicking on the arrow in the recent. The data is in the key value pair, one point t
9. The admin team will have to do the field extractions, only the key value pairs are extracted and made int he proper format. Field extractions to be done, manual extractions base done expressions limit the filtering on the later part of the search- so as a best practice do as must as search and filter in the base search as possible
10. Key word search example “error” keyword is given to Splunk- it searches across all events for the keyword
11. text from the pdf when updated may not format as expected. for format- properly – control | is used to format the results.
12. Table commands- the field mentioned will allow you to see the results in the format of a table
13. fields command- to remove fileds or order fields in a particular way
14. top- finds the most common values of the given field and % distribution and count
15. top is easy to make the visualization- in the results just cluck o visualization from the data searched.
16. rare – opposite to top
17. Stats – enables users to calculate the statistics
18. Sum –
19. As –
20. Group by – count by
21. eval – used to create an extra column with a default value or a formula evaluation of the values. ex: eval abdc=if(x<‘5000, 8000, abdc) 
22. Time chart- takes results and formulate in the time selected in the time picker.
23. span – can be used in time chart command to chunk the time intervals- for trends etc.
24. Stats –
25. Base search – the search before the pipe, mostly index, source, source type, host.
26. transforming search – everything after the pipe | symbol written after the base search

Search Processing language

1. wild card- * ex: fail* leads to search of failed or failure or fails – used after the string is more efficient than at front.
2. AND NOT OR –
   1. ex: failed password is like failed AND password
   2. ex: failed OR password displays all combinations
   3. Order is NOT OR AND
   4. parenthesis is used to control the order of evaluation
   5. “Failed password ” in general used with quotes to search

Features and terms used on Splunk on day 2-day use!

• Shared Search jobs
• Export Results- raw- csv, xml, Json
• Search mode- fast (no field discovery)
• verbose- discovering all data as can
• default mode- –
• Timeline- visual rep of segments on the time- on clicking the timeline- we ca see the event generates on that time.

What is an event? – time index- based on time zone in user account bottom row has the selected fields, rolling

Other factors used can be noted below:

Add to Search

icon- to open in new browse window

Clicking on highlighted text can add or remove to search

event actions

field actions

Search Processing language

Wild cards – *

search terms are not case sensitive

AND OR NOT can be used for multiple familiar words like US or CA

Order Role evaluation

not or and (Preference)

“

\”

What are commands, functions, clauses, arguments in search terms?

how we want to search- a site’s foundation of search queries.

Commands -what we need to do with the searches results- create charts, computing statistics and formatting

Functions – explains how we want to compute and evaluate the result

Arguments – variables we need to apply for the functions

Clauses – how we want results (group or defined)

Below terms can be used in the search

Index

host

Source type

Stats

Count

visits

search Visits >1

There are certain admin consoles not all the users may have access to.

Splunk Specialist with good IT infrastructure skills, in multi-platform environments, ideally familiar with Linux. There are several innovative projects in Splunk, and various companies are looking for qualified administrators with Splunk experience and/or certification.

Main responsibilities:

• Participated in all Splunk company initiatives, both internal projects and customer mandates.
• Install and configure the necessary components to collect data from DB, log files, API, etc. to Splunk.
• Install, configure, administer Splunk Enterprise on Windows and Linux.
• Support Splunk updates.
• Monitor and identify performance issues.
• Perform data onboarding in Splunk: data collection, filtering, and transformation (source types, inputs, transforms, etc.);
• Build use cases: advanced SPL, dashboards, reports, alerts, etc.
• Always continue to develop product knowledge and act as a product expert.
• Document best practices.

Qualifications required:

• Integrating data from various sources (DB, log files, APIs, etc.) into Splunk (on prem or cloud);
• Experience in CIM modeling in Splunk.
• Experience in managing indexes and knowledge objects in Splunk.
• Experience working with cloud offerings such as Azure or AWS.
• Knowledge of basic security concepts.
• Experience in access management (RBAC model) in Splunk.
• Valuable experience in AIX, Linux (RedHat, CentOS) systems administration (permissions management, security (including TLS/SSL), debugging, etc.);
• Exceptionally good experience in Splunk user support and training.
• Good knowledge of system virtualization.
• Good knowledge of server infrastructure.
• Knowledge of storage, operating systems and networking.
• Knowledge of Splunk Enterprise Security is an asset.