Skip to content

Commit

Permalink
refactor: include artifacts (#169)
Browse files Browse the repository at this point in the history
* refactor: include artifacts
  • Loading branch information
cugu authored Oct 19, 2024
1 parent caade8d commit cffa0e3
Show file tree
Hide file tree
Showing 21 changed files with 3,020 additions and 40 deletions.
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ jobs:
- run: make install generate-win fmt_linux
- run: git diff --exit-code

validate:
name: validate
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v5
with: { go-version: '1.23' }
- uses: actions/checkout@v4
- run: make validate

lint:
name: lint
runs-on: ubuntu-latest
Expand Down
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# config
config/artifacts/

# go
vendor

Expand Down
23 changes: 6 additions & 17 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,17 @@ test-coverage:
go tool cover -func=coverage.out
go tool cover -html=coverage.out

.PHONY: validate
validate:
@echo "Validating..."
cd tools/artifactvalidator && go build -o ../../build/bin/artifactvalidator .
./build/bin/artifactvalidator -entrypoints=DefaultCollection1 config/artifacts/*.yaml

.PHONY: generate
generate:
@echo "Generating..."
go install golang.org/x/tools/cmd/goimports@v0.1.7
go install github.com/forensicanalysis/go-resources/cmd/resources@v0.4.0
rm -rf config/artifacts
git clone https://github.com/forensicanalysis/artifacts.git config/artifacts
go run tools/yaml2go/main.go config/ac.yaml config/artifacts/*.yaml
resources -package assets -output assets/bin.generated.go config/bin/*

Expand All @@ -68,21 +72,6 @@ generate-win: generate
rsrc -arch amd64 -manifest build/win/artifactcollector.exe.user.manifest -ico build/win/artifactcollector.ico -o build/win/artifactcollector.user.syso
rsrc -arch 386 -manifest build/win/artifactcollector32.exe.user.manifest -ico build/win/artifactcollector.ico -o build/win/artifactcollector32.user.syso

.PHONY: build
build: generate
@echo "Building..."
go build -o build/bin/artifactcollector .

.PHONY: build-linux
build-linux: generate
@echo "Building for Linux..."
GOOS=linux GOARCH=amd64 go build -o build/bin/artifactcollector-linux .

.PHONY: build-darwin
build-darwin: generate
@echo "Building for macOS..."
GOOS=darwin GOARCH=amd64 go build -o build/bin/artifactcollector-darwin .

.PHONY: build-win
build-win: generate-win
@echo "Building for Windows..."
Expand Down
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,18 +61,18 @@ The zip file contains the results of the extraction and needs to be transferred
## Build your own artifactcollector

1. Clone the repository: `git clone https://github.com/forensicanalysis/artifactcollector`.
2. Add artifact definition yaml files as needed in `config/artifacts`. Do not edit the
artifact definitions, as they will be overwritten.
2. Add and edit artifact definition yaml files as needed in `config/artifacts`.
3. Edit `config/ac.yaml` and add the artifacts you want to collect.
4. On windows, you can move the syso into the root folder (e.g. `cp resources\artifactcollector.syso .`)
to enable the icon for the executable and the UAC popup.
5. Run `make build` to generate the artifactcollector binary.
5. Run `go build .` to generate the artifactcollector binary.
1. You can also use `GOOS=windows GOARCH=amd64 go build -o artifactcollector.exe .` to cross-compile for Windows.

## Embed binaries

Binaries can be added to `config/bin` and then included into the artifactcollector
in the `make build` step. Additionally, a corresponding COMMAND artifact like
the following is required.
in the `go build` step. Additionally, a corresponding COMMAND artifact like
the following is required:

```yaml
name: Autoruns
Expand All @@ -84,12 +84,19 @@ sources:
supported_os: [ Windows ]
```
The command output to stdout and stderr is saved, but generated
files are not collected.
The command output to stdout and stderr is saved, but generated files are not collected.
## Acknowledgement
The artifactcollector uses on the following great projects:
- [config/artifacts](config/artifacts) is based on the awesome [Forensic Artifacts](https://github.com/ForensicArtifacts/artifacts) project.
- [doublestar](doublestar) is based on [Bob Matcuk's](https://github.com/bmatcuk) great [doublestar](https://github.com/bmatcuk/doublestar) package.
- [store/aczip](store/aczip) and [build/go](build/go) contain code from the Go standard library.
## License
Most of the artifactcollector is licensed under the MIT License. See [MIT license](LICENSE) for the full license text.
The directories [store/aczip](store/aczip) and [build/go](build/go) contain code from the Go standard library
which is licensed under the [BSD-3-Clause license](LICENSE-BSD).
which is licensed under the [BSD-3-Clause license](LICENSE-BSD).
2 changes: 1 addition & 1 deletion assets/artifacts.generated.go

Large diffs are not rendered by default.

2 changes: 0 additions & 2 deletions build/win2k/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ WORKDIR /repo
RUN go install golang.org/x/tools/cmd/goimports@v0.1.7
RUN go install github.com/forensicanalysis/go-resources/cmd/resources@v0.4.0
RUN go install github.com/akavel/rsrc@v0.10.2
RUN rm -rf config/artifacts
RUN git clone https://github.com/forensicanalysis/artifacts.git config/artifacts
RUN go run tools/yaml2go/main.go config/ac.yaml config/artifacts/*.yaml
RUN resources -package assets -output assets/bin.generated.go config/bin/*
RUN rsrc -arch amd64 -manifest build/win/artifactcollector.exe.manifest -ico build/win/artifactcollector.ico -o build/win/artifactcollector.syso
Expand Down
2 changes: 0 additions & 2 deletions build/winxp/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ WORKDIR /repo
RUN go install golang.org/x/tools/cmd/goimports@v0.1.7
RUN go install github.com/forensicanalysis/go-resources/cmd/resources@v0.4.0
RUN go install github.com/akavel/rsrc@v0.10.2
RUN rm -rf config/artifacts
RUN git clone https://github.com/forensicanalysis/artifacts.git config/artifacts
RUN go run tools/yaml2go/main.go config/ac.yaml config/artifacts/*.yaml
RUN resources -package assets -output assets/bin.generated.go config/bin/*
RUN rsrc -arch amd64 -manifest build/win/artifactcollector.exe.manifest -ico build/win/artifactcollector.ico -o build/win/artifactcollector.syso
Expand Down
3 changes: 3 additions & 0 deletions config/ac.yaml
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
artifacts: ["DefaultCollection1"] # artifact definitions to collect
user: false # optional, if true, do not request admin permissions
case: "" # optional case name
output_dir: "" # optional output directory
14 changes: 14 additions & 0 deletions config/artifacts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## Artifact Definitions

The artifactcollector uses yaml files to define forensic artifacts it can collect.

The yaml files are based on the [ForensicArtifacts/artifacts](https://github.com/ForensicArtifacts/artifacts)
repository, but with the following major changes:

- `provides` on source level are added to enable extraction of parameters
- All source types are distinctly defined, including the `DIRECTORY` type.
- Parameter expansion and globing is defined, including `**`.
- Inconsistent trailing `\*` in REGISTRY_KEYs are removed.

The [Style Guide](style_guide.md) describes the full specification of the artifact definitions
how they are used in the artifactcollector.
168 changes: 168 additions & 0 deletions config/artifacts/collections.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Predefined opinionated collections

name: DefaultCollection1
doc: Predefined opinionated collections
sources:
- type: ARTIFACT_GROUP
attributes:
names:
- 'FOR500'
- 'WindowsComputerName'
- 'WindowsEventLogs'
- 'WindowsHotfixes'
- 'WindowsNetworkInterfaceInformation'
- 'WindowsPersistence'
- 'WindowsRunKeys'
- 'WindowsServices'
- 'WindowsUninstallKeys'
- 'WindowsUSBInformation'
supported_os: [ Windows ]
- type: ARTIFACT_GROUP
attributes:
names:
# - 'BrowserCache'
- 'BrowserHistory'
- 'LinuxIPTablesRulesCommand'
- 'LinuxAtJobsFiles'
- 'LinuxAuditLogFiles'
- 'LinuxCronTabFiles'
- 'LinuxHostnameFile'
supported_os: [ Linux ]
- type: ARTIFACT_GROUP
attributes:
names:
# - 'BrowserCache'
- 'BrowserHistory'
- 'MacOSAtJobsFile'
- 'MacOSAuditLogFiles'
- 'MacOSBashHistoryFile'
- 'MacOSCronTabFile'
- 'MacOSHostsFile'
- 'MacOSLastlogFile'
- 'MacOSMiscLogFiles'
- 'MacOSRecentItemsFiles'
- 'MacOSSystemLogFiles'
- 'MacOSUserTrashFiles'
supported_os: [ Darwin ]
supported_os: [ Darwin,Linux,Windows ]
---
# Artifacts from the SANS FOR500 course

name: FOR500
doc: Windows Forensic Analysis
sources:
- type: ARTIFACT_GROUP
attributes:
names:
- WindowsBrowserArtifacts
- WindowsProgramExecution
- WindowsDeletedFiles
- WindowsNetworkActivity
# - WindowsFileOpening
- AccountUsage
- ExternalDevice
supported_os: [ Windows ]
---
name: WindowsBrowserArtifacts
doc: WindowsBrowserArtifacts
sources:
- type: ARTIFACT_GROUP
attributes:
names:
- WindowsOpenSaveMRU
- WindowsOpenSavePidlMRU
# EmailAttachments
# SkypeMainDirectory is only for macos
# BrowserCache not collected by default
- BrowserHistory
# AdsZoneIdentifier
supported_os: [ Windows ]
---
name: WindowsProgramExecution
doc: Program Execution
sources:
- type: ARTIFACT_GROUP
attributes:
names:
# UserAssist
- WindowsActivitiesCacheDatabase
- WindowsMostRecentApplication
- WindowsAppCompatCache # Shimcache
# JumpLists
- WindowsAMCacheHveFile
- WindowsSystemResourceUsageMonitorDatabaseFile
# BAM/DAM
# LastVisitedMRU
- WindowsPrefetchFiles
supported_os: [ Windows ]
---
name: WindowsDeletedFiles
doc: Deleted Files
sources:
- type: ARTIFACT_GROUP
attributes:
names:
# ACMRU
# Thumbcache
# Thumbs.db
# IEEdgeFile -> WindowsBrowserArtifacts
# WordWheelQuery
- WindowsRecycleBin
# LastVisitedMRU -> WindowsProgramExecution
supported_os: [ Windows ]
---
name: WindowsNetworkActivity
doc: Network Activity
sources:
- type: ARTIFACT_GROUP
attributes:
names:
- WindowsTimezone
- InternetExplorerCookiesFile
# NetworkHistory
# WLANEventLog
# BrowserSearchTerms -> WindowsBrowserArtifacts
# WindowsSystemResourceUsageMonitorDatabaseFile -> WindowsProgramExecution
supported_os: [ Windows ]
# ---
# name: WindowsFileOpening
# doc: File Opening
# sources:
# - type: ARTIFACT_GROUP
# attributes:
# names:
# # WindowsOpenSaveMRU -> WindowsBrowserArtifacts
# # RecentFiles
# # JumpLists -> WindowsProgramExecution
# # ShellBags
# # LNKFiles
# # WindowsPrefetchFiles -> WindowsProgramExecution
# # LastVisitedMRU -> WindowsProgramExecution
# # IEEdgeFile -> WindowsBrowserArtifacts
# # OfficeRecentFiles
# supported_os: [Windows]
---
name: AccountUsage
doc: Account Usage
sources:
- type: ARTIFACT_GROUP
attributes:
names:
- WindowsSystemRegistryFiles
- WindowsXMLEventLogSecurityFile
supported_os: [ Windows ]
---
name: ExternalDevice
doc: External Device
sources:
- type: ARTIFACT_GROUP
attributes:
names:
# KeyIdentification
- WindowsSetupApiLogs
# User
# PnPEvents
# VolumeSerialNumber
# DriverLetter
# LNKFiles -> WindowsFileOpening
supported_os: [ Windows ]
65 changes: 65 additions & 0 deletions config/artifacts/linux.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Linux specific artifacts.

name: LinuxIPTablesRulesCommand
doc: List IPTables rules.
sources:
- type: COMMAND
attributes:
args: [ "-L", "-n", "-v" ]
cmd: /sbin/iptables
supported_os: [ Linux ]
---
name: LinuxAtJobsFiles
doc: Linux at jobs.
sources:
- type: FILE
attributes: { paths: [ '/var/spool/at/*' ] }
supported_os: [ Linux ]
---
name: LinuxAuditLogFiles
doc: Linux audit log files.
sources:
- type: FILE
attributes: { paths: [ '/var/log/audit/*' ] }
supported_os: [ Linux ]
---
name: LinuxCronTabFiles
doc: Crontab files.
sources:
- type: FILE
attributes:
paths:
- '/etc/crontab'
- '/etc/cron.d/*'
- '/var/spool/cron/**'
supported_os: [ Linux ]
---
name: LinuxHostnameFile
doc: Linux hostname file.
sources:
- type: FILE
attributes: { paths: [ '/etc/hostname' ] }
supported_os: [ Linux ]
---
name: LinuxPasswdFile
doc: |
Linux passwd file.
A passwd file consist of colon separated values in the format:
username:password:uid:gid:full name:home directory:shell
sources:
- type: FILE
attributes: { paths: [ '/etc/passwd' ] }
provides:
- key: users.homedir
regex: '.*:(.*?):.*'
supported_os: [ Linux ]
---
name: LinuxHomePath
doc: Users directories in /home
sources:
- type: PATH
attributes: { paths: [ '/home/*' ] }
provides:
- key: users.homedir
supported_os: [ Linux ]
Loading

0 comments on commit cffa0e3

Please sign in to comment.