forked from Velocidex/velociraptor-docs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
1. Preserving Forensic Evidence 2. Triaging Logs
- Loading branch information
Showing
12 changed files
with
368 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
--- | ||
menutitle: "Playbooks" | ||
title: "Playbooks" | ||
date: 2024-06-12T14:03:59Z | ||
draft: false | ||
weight: 150 | ||
pre: <i class="fas fa-play"></i> | ||
no_edit: true | ||
disableToc: false | ||
no_children: true | ||
noDisqus: true | ||
outputs: | ||
- html | ||
- RSS | ||
--- | ||
|
||
Velociraptor is an incredibly powerful tool, but sometimes it is hard | ||
to know where to start. This page aims to help newcomers to | ||
Velociraptor by presenting a set of playbooks to use when faced with | ||
certain tasks. | ||
|
||
{{% children description=True %}} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,206 @@ | ||
--- | ||
title: "Preserving Forensic Evidence" | ||
description: | | ||
A compromised endpoint is likely to be destroyed. You want to preserve raw files until you have an opportunity to analyse them later. | ||
tags: | ||
- preservation | ||
- acquisition | ||
--- | ||
|
||
## Scenario | ||
|
||
As a system administrator you have a high level of confidence a | ||
certain endpoint is compromised. You wish to preserve critical | ||
evidence while arranging for a more experienced DFIR professional to | ||
examine the evidence. | ||
|
||
## Main takeaways | ||
|
||
1. This technique is a "shotgun" approach - it typically collects a | ||
lot of data. | ||
2. Not very targeted. | ||
3. Does not scale to many endpoints - use when very confident an | ||
endpoint is compromised. | ||
4. DFIR skill required - LOW. | ||
|
||
## Steps to take | ||
|
||
Some common artifacts that are used in a preservation collection: | ||
|
||
1. `Windows.KapeFiles.Targets` - collects files such as `$MFT`, log files etc. | ||
2. `Windows.Memory.Acquisition` - collects a physical memory image. | ||
|
||
### If Velociraptor is already installed on the endpoint | ||
|
||
For collecting from a single endpoint | ||
|
||
1. Search for the client you want to acquire from the main search screen. | ||
2. Select `Collected Artifacts` from the sidebar, then select `New | ||
Collection` from the toolbar. | ||
3. For Windows systems, select `Windows.KapeFiles.Targets`. | ||
4. Select the `BasicCollection` or `Kape Triage` or `SANS Triage` | ||
targets. These targets select the most common raw files used by | ||
DFIR analysts. | ||
5. This artifact can collect a large amount of data, we recommend to | ||
place resource limits to ensure that collection is not too | ||
excessive. | ||
|
||
![Selecting the Windows.KapeFiles.Targets artifact](kape_targets.png) | ||
|
||
If you will have a number of endpoints to collect you can use a hunt | ||
to combine the data together: | ||
|
||
1. In the `Hunt Manager` screen, create a new hunt. Restrict the hunt | ||
to a label `acquisition`. | ||
2. Proceed to collect the same artifacts as above | ||
3. As the investigation proceeds, when you find a client that warrants | ||
a preservation collection, simply add a label `acquisition` to the | ||
client. The `Windows.KapeFiles.Targets` will be automatically | ||
scheduled on this client. | ||
4. You can view all acquired clients in the hunt overview page, and | ||
create a large zip export of all preserved files. | ||
|
||
{{% notice warning "Ensure the hunt is restricted by labels!" %}} | ||
|
||
Make sure this hunt is restricted by labels! If it is not, it will be | ||
scheduled on all clients and may result in a lot of data being | ||
transferred to the server - this can result in the server's bandwidth | ||
being saturated, disk becoming full or server becoming unresponsive. | ||
|
||
If this happens you can stop the hunt in the GUI which will cancel all | ||
in-flight collections and eventually recover the server's bandwidth. | ||
|
||
{{% /notice %}} | ||
|
||
### If Velociraptor is not installed on the endpoint | ||
|
||
If Velociraptor is not already installed on the endpoint, you can use | ||
an `offline collector`. The Velociraptor `Offline Collector` is a | ||
preconfigured version of Velociraptor which can run on the endpoint | ||
and write all raw files into a ZIP archive. | ||
|
||
![Creating an offline collector](offline_collector.png) | ||
|
||
You can use a number of methods to push the offline to collection to | ||
the endpoint: | ||
|
||
1. Use an existing EDR or endpoint security product to run the binary | ||
and retrieve the collected ZIP file. | ||
2. Use Group Policy to schedule a task that runs the collector from a | ||
remote share. | ||
3. Use `WinRM` or `PsExec` to run the collector remotely (by careful | ||
of pass the hash type attacks though). | ||
|
||
We recommend using the X509 encryption scheme to store the raw data to | ||
an encrypted container. This helps to protect it in transit. | ||
|
||
![Encrypting the collected data](encryoted_collector.png) | ||
|
||
You can also configure the offline collector to automatically upload | ||
the collection to a cloud storage, such as `S3`, `Google Cloud | ||
Storage` or a `Windows file share`. | ||
|
||
## When to use this technique? | ||
|
||
This technique should be used sparingly, usually targeting few | ||
systems, as it can transfer a large amount of data. | ||
|
||
Typically this technique is used when the endpoint is likely to be | ||
destroyed (e.g. re-imaged) or become inaccessible in the near | ||
future. The method is most suitable for preservation of raw data. | ||
|
||
## When not to use this technique? | ||
|
||
This workflow is not appropriate for triage. A triage workflow is | ||
about discovering unknown compromised endpoints and therefore by | ||
definition must be applied to a large number of endpoints. The | ||
`Preserving Forensic Evidence` workflow can not be applied to a large | ||
number of endpoints. | ||
|
||
The traditional digital forensics workflow consists of the following | ||
steps: | ||
|
||
1. `Acquisition` step collects a large amount of raw data to a central | ||
server. | ||
2. `Analysis` step applies parsers and various dedicated tools in a | ||
central "forensics workstation" on all the raw data collected. | ||
3. `Interpretation` step involves examining the output from the | ||
various parsers to answer case specific questions. | ||
|
||
Many newcomers to Velociraptor have been trained with this workflow | ||
and try to apply it to Velociraptor. | ||
|
||
However this centralist approach to digital forensics does not scale | ||
in practice. When trying to apply this approach to a large network of | ||
endpoints, users are often overwhelmed with data (it is almost a right | ||
of passage for new Velociraptor users to completely fill their | ||
server's disk because they tried to collect all the log files from all | ||
the endpoints) | ||
|
||
Velociraptor's philosophy is different: | ||
|
||
1. Velociraptor considers the endpoint to be the ultimate source of truth. | ||
2. Therefore we rarely need to fetch the raw data from the endpoint! | ||
Since Velociraptor can parse the raw data directly on the endpoint | ||
we prefer asking case relevant questions from the entire network | ||
directly. | ||
|
||
To explain this point, let's consider an example where we want to know | ||
if an executable file was recently downloaded onto the endpoint. We | ||
will attempt to parse the USN Journal to answer this question. | ||
|
||
The very traditional Digital Forensics process (i.e. back in the 90s) | ||
would require acquiring a bit for bit disk image of the suspected | ||
endpoint with a write blocker attached! Probably no-one does this any | ||
more! | ||
|
||
A more modern approach is to use a digital acquisition tool to copy | ||
just the `C:\$Extend\$UsnJrnl:$J` file in it's entirety for | ||
`Acquisition`, then apply a separate tool to parse the journal and | ||
search for any files written to the disk with a `.exe` extension. | ||
|
||
While this approach is better than acquiring the entire disk image | ||
(Average disks are of the range of 1-2Tb these days), it still | ||
requires collecting several hundred MB of data from each endpoint. If | ||
you have 10,000 endpoints this quickly becomes intractable. | ||
|
||
Velociraptor's approach is different: Since the endpoint is considered | ||
the source of truth, we just directly run our analysis on the | ||
endpoint. Collecting the `Windows.Forensics.Usn` artifact with a | ||
`FileNameRegex` filter of `.exe$` will query each machine to parse | ||
their own local copy of the USN journal to find all executables | ||
directly. | ||
|
||
We can collect this artifact using a hunt from the entire network at | ||
once. Since each endpoint will be doing the analysis in parallel, | ||
results will return in minutes regardless of how many endpoint there | ||
are! | ||
|
||
This is why we consider simple acquisition followed by central | ||
analysis workflows to be inferior to directly collecting the right | ||
artifacts. | ||
|
||
You should use the `Preserving Forensic Evidence` workflow only on | ||
systems that are known to have been compromised and you want to | ||
preserve the raw files for some reason. | ||
|
||
### Other considerations | ||
|
||
When collecting raw files from the endpoint we need to make a tradeoff: | ||
|
||
1. Collecting fewer files may miss some files that are needed later | ||
during analysis. | ||
2. Collecting more files will result in smaller collection. | ||
|
||
Ultimately it is not possible to know in advance what type of evidence | ||
may be relevant. For example, we might collect the `$MFT` in our | ||
initial acquisition but subsequent analysis can show that certain | ||
files are critical (e.g. files in the user's `Downloads` directory). | ||
|
||
In a `Preserving Forensic Evidence` workflow we can not go back to the | ||
source of truth and check the files in the `Downloads` directory! | ||
|
||
Therefore it is always better to have Velociraptor already installed | ||
on the endpoint so we can pivot during the analysis phase and get | ||
additional information as required. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,140 @@ | ||
--- | ||
title: "Triaging Logs" | ||
description: | | ||
An endpoint is suspected of being compromised but you dont know | ||
exactly what happened. You want to get an initial idea by examining | ||
the logs on the actual endpoint. | ||
tags: | ||
- triage | ||
- preservation | ||
--- | ||
|
||
## Scenario | ||
|
||
An endpoint is suspected of being compromised but you dont know | ||
exactly what happened. You want to get an initial idea by examining | ||
the logs on the actual endpoint. | ||
|
||
## Main takeaways | ||
|
||
1. This technique is similar to forwarding logs to a SIEM and applying | ||
signatures. | ||
2. However we can choose very noisy signatures here | ||
3. We use stacking to quickly categorize the types of activity that | ||
happens on the endpoint. | ||
|
||
## Steps to take | ||
|
||
Some common artifacts that are used for `Triaging Logs` | ||
|
||
1. `Windows.Hayabusa.Rules` should be imported using `Server.Import.CuratedSigma` | ||
2. `Exchange.Windows.EventLogs.Hayabusa` should be imported from the | ||
artifact exchange. | ||
|
||
### Importing Windows.Hayabusa.Rules | ||
|
||
1. Select the `Server Artifacts` from the sidebar. | ||
2. Add a collection, search for `Server.Import.CuratedSigma` and | ||
import the `Windows.Hayabusa.Rules`. This will import the latest | ||
version of the artifact. | ||
|
||
This artifact uses the built in `Sigma Engine` in Velociraptor. The | ||
artifact packages the curated `Hayabusa` rules in a convenient | ||
artifact. Rules are categorized by `RuleLevel` and `RuleStatus` which | ||
generally try to balance how noisy a rule against its detection | ||
efficacy. | ||
|
||
![Sigma workflow in Velociraptor](velociraptor_sigma_flow.png) | ||
|
||
Because we are trying to triage the endpoint, we actually want to see | ||
all the hits, even if they are noisy. We will apply stacking later to | ||
quickly triage the type of activity on the endpoint. So in this case | ||
we should select to apply all the rules. | ||
|
||
![Applying all Sigma Rules](all_rules.png) | ||
|
||
Once the artifact is collected from the endpoint we can stack the hits | ||
in the GUI: | ||
|
||
1. Update the notebook to remove the `LIMIT 50`. This will select all | ||
rows in one table. Typically there should be many thousands of | ||
rows because we added all the noisy rules. | ||
2. Sort by the rule `Title`. Hover the mouse on the column header and | ||
click the `Sort` button. | ||
3. Once the column is sorted, the stacking button should appear. | ||
|
||
![Stacking a column](stacking_a_column.png) | ||
|
||
4. Clicking the stacking button will show a summary of the different | ||
rules matching and a count of how many times each rule made a hit. | ||
|
||
![Viewing column summaries](viewing_column_stack.png) | ||
|
||
5. Clicking on any of the rules navigates the table to the list of | ||
rules that actually hit. | ||
|
||
![Viewing common rows](viewing_common_rows.png) | ||
|
||
Using this technique it is possible to quickly identify the types or | ||
categories of activity on the endpoint and see the most suspicious | ||
rules. Due to the stacking we dont need to review each individual hit, | ||
but only the different types of rules. | ||
|
||
For example, say we see a rule description a `PsExec` lateral | ||
movement, we can quickly identify if `PsExec` is expected for this | ||
environment, or does it represent a potential threat. If I identify | ||
the rule as important, I can then review each instance to get more | ||
information about what commands were run. | ||
|
||
## Using the Exchange.Windows.EventLogs.Hayabusa | ||
|
||
The `Exchange.Windows.EventLogs.Hayabusa` artifact is available in the | ||
artifact exchange. This artifact uses an external binary `Hayabusa` to | ||
evaluate the `Sigma` rules from the `Hayabusa` project. | ||
|
||
Post processing and analysing the results from this artifact is | ||
similar to the procedure described above. | ||
|
||
|
||
## Discussion and limitations | ||
|
||
This technique is similar to many SIEMs which forward event logs from | ||
the endpoint and apply matching rules. There are some fundamental | ||
differences though: | ||
|
||
1. SIEMs typically only forward a small subset of logs since the more | ||
logs are collected the more data the SIEM backend needs to | ||
handle. Typically SIEMs forward logs such as `Sysmon Process | ||
Execution` but do not forward other logs for example `BITS Client | ||
Operational Logs`. | ||
|
||
2. SIEM rules are also written to ensure they have a low false | ||
positive rate. This means that suspicious activity in one | ||
environment which is common in another setting, might not trigger a | ||
detection. By stacking on all noisy rules we get to decide for | ||
ourselves if a particular rule is acceptable for this environment. | ||
|
||
For example an administrator RDP login may be perfectly normal in | ||
some environments but a red flag in others! SIEM detections are | ||
rarely tuned to the environment. | ||
|
||
3. A SIEM may not be present or well tuned, in a particular | ||
environment. Running the `Sigma` log triaging workflow can | ||
compensate for the lack of a SIEM. | ||
|
||
4. Data retention is different from a SIEM. Typically SIEMs only | ||
maintain logs for limited time (sometimes as low as a month). On | ||
the other hand log files are typically rotated based on size. This | ||
means that sometimes logs will be present on the endpoint for | ||
longer than in the SIEM, while other times the SIEM will contain | ||
logs that were already rotate on the endpoint. | ||
|
||
5. Because this technique relies on locally stored logs, it is | ||
susceptible to logs being cleared by attackers. | ||
|
||
## Real time monitoring | ||
|
||
As an additional step you can enable the `Windows.Hayabusa.Monitoring` | ||
artifact for real time detection of the same `Sigma` rules. This can | ||
provide coverage for any future compromises |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.