Support Empire for system-wide deployment

[D3vil0p3r]

Describe your changes

Refactored code to make Empire to be deployed at system-level (multi-user) with no impact on the current approach based on working only in a case Empire is cloned in a user-granted directory.

This PR makes Empire working also in case it is installed in a system directory without changing the current grants. It is possible by refactoring code by following XDG Base Dir specifications.

Issue ticket number and link (if there is one)

#756

Checklist before requesting a review

• I have performed a self-review of my code
• If it is a core feature, I have added thorough tests.
• I have added an entry to CHANGELOG.md
• I have updated the documentation in docs/ (if applicable)

[vinnybod]

The reason the run_as_user function exists is because users are likely to run empire with sudo since it is required to open lower port numbers, but we don't want log files, downloads, cloned submodules, etc to end up with root user permissions.

For example, this function prevents the installation and the update of submodules and Starkiller because it "removes" privileges to the running sudo user.

Can you explain this in a bit more detail? You shouldn't need sudo to update the starkiller submodule.

[D3vil0p3r]

Can you explain this in a bit more detail? You shouldn't need sudo to update the starkiller submodule.

Hey. sudo is not needed if I clone the Empire repository in my HOME folder, so in the case Empire is installed for only the current user. If an admin would like to install Empire at system-level (all users use case), it is needed to use sudo to get/update starkiller submodule, because it will be written in a system-level directory (i.e., /usr/share/Empire) where the standard user won't have grants.

[D3vil0p3r]

@vinnybod any clue about my last comment?

[vinnybod]

The current assumption Empire makes is that it is:

• not installed as admin
• run as admin for opening ports
• doesn't write files as admin

I'd approve a PR that adds your admin install use case as long as it still supports the above assumptions too.

[D3vil0p3r]

I need to do with you a deeper analysis to properly work on the code.

Problem statement

Empire cannot work at system-level, for example when on my host I have 10 user accounts (i.e., in a school), I cannot install one single Empire instance for everyone but I need to install N instances for each user by consuming a lot of resources.

Your requirements

• not installed as admin
• run as admin for opening ports
• doesn't write files as admin

not installed as admin

• Why don't you need Empire installed as root? I didn't get well your first comment above.

run as admin for opening ports

So, opening ports on the local network, and admin is needed.

• Currently is this done right?
• Is it done by asking user to run Empire as sudo?
• In which code section are they opened?

doesn't write files as admin

• Why?
• Can you provide me a detailed motivation?
• What are the files that must not be written as admin? And on which code section are used?

run_as_user() scope

Currently run_as_user() is used in:

• cloning starkiller
• fetch, checkout and pull Empire git repository
• fetch submodules of Empire git repository

So, currently, run_as_user() function keeps these ONLY 3 operations above as low privilege even if the user uses sudo. But if Empire has other operations that write on the system, they should be still written as root

My proposal

I don't see relevant risks to install Empire at system-level (so as root) and to pull updates and submodules. If you want to be compliant with the minimum privilege principles, what we can do is to allow Empire to be installed as root at system-level (as occurs for any tools/packages in Linux), but we can identify all those files, resources, elements that must be dealt as current user instead of root, and manage them in the user HOME folder.

In this context run_as_user() can be used to exclusively deal with those files created by Empire. And we can create an additional function run_as_sudo() used only to update repository and submodules if Empire was installed at system-level, and for opening the needed ports.

[vinnybod]

Why don't you need Empire installed as root? I didn't get well your first comment above.
I don't see relevant risks to install Empire at system-level (so as root) and to pull updates and submodules. If

It's more of a convenience factor for us at maintainers. There have been a lot of issues opened in the past due to file permissions issues. So enforcing installs and the file writes to be user-level decreases that.

For example:
git repo was cloned without sudo, but submodules were pulled with sudo causing future git commands to fail due to mixed file ownership
Running empire within an IDE for development fails due to the log files requiring elevated write access due to previously being written to as sudo

So, opening ports on the local network, and admin is needed.
Currently is this done right?
Is it done by asking user to run Empire as sudo?
In which code section are they opened?

We handle this automatically with the ps-empire command
http listeners bind to ports and many users use priveleged ports (<1024)

doesn't write files as admin
Why?
Can you provide me a detailed motivation?
What are the files that must not be written as admin? And on which code section are used?

Sort of got into this above, but ideally all the files in the application directory would have uniform access permissions.

It is less about risks and more about alleviating inbound requests related to issues stemming from people installing in different ways. My intention is that for 90%+ of users, the standard install that we enforce "just works".
We'd still want to unblock the minority, but not at the expense of making the "easy" case less functional.

[D3vil0p3r]

I try to split the problem in small problems because involving all users could be complex but still possible. If we want to try to "unblock" the minority of users, and to not impact the current 90+% users, I would start to manage from the raising issues when Empire is installed system-wide BUT with non-root run. I will edit the PR code.

1. Issue: Permission denied on directories specified in config files

empire-server

Traceback (most recent call last):
  File "/usr/share/empire/empire.py", line 13, in <module>
    server.run(args)
  File "/usr/share/empire/empire/server/server.py", line 161, in run
    setup_logging(args)
  File "/usr/share/empire/empire/server/server.py", line 38, in setup_logging
    log_dir.mkdir(parents=True, exist_ok=True)
  File "/usr/lib/python3.12/pathlib.py", line 1311, in mkdir
    os.mkdir(self, mode)
PermissionError: [Errno 13] Permission denied: '/usr/share/empire/empire/server/downloads/logs'

Solution: code will check config.yaml file for the path of server and client. If the specified logs path is at user-level, it creates as-is; else... fallback to the standard user directory empire logs path $HOME/.empire/.

This approach is compliant to Linux specs (XDG Base Directory Specification that standardizes where application files, including logs, should be stored within the user’s home directory.)

Advantages:

• Standardized approach
• Empire usage for multi-user systems
• Multiplatform (Linux, Darwin, Windows)
• Not impact on the current Empire permission approach discussed above

2. Add check on fetch_submodules()

Added:

def fetch_submodules():
    if not os.path.exists(Path(".git")):
        log.info("No .git directory found. Skipping submodule fetch.")
        return
    command = ["git", "submodule", "update", "--init", "--recursive"]
    run_as_user(command)

to run this function only if .git has been found. It is the same logic you implemented for check_submodules().

3. config.yaml must be copied in HOME user folder

In Linux/Windows, configuration files must be copied from the default one to HOME folder, in order to allow it to manage according XDG standard specification. It allows to cover both user-level and system-level approach.

Furthermore, the definition of config file path must be managed centrally.

4. Manage dirs specified in config.yaml to fallback to $HOME directory - WIP

server/config.yaml has:

directories:
  downloads: empire/server/downloads/
  module_source: empire/server/data/module_source/
  obfuscated_module_source: empire/server/data/obfuscated_module_source/

I would suggest to manage the last two paths in a fixed manner on the code by removing them from config file, because Empire already provides all those modules. If a user changes those two lines, Empire starts to trigger errors because cannot find its directory where stored modules. If a user wants to add a custom module, Empire docs can report that user can add it in <root-empire-dir>/empire/server/data/{module_source, obfuscated_module_source} directory.

[vinnybod]

@D3vil0p3r thanks for iterating on this with me 😅 . I'm still learning the best practices around Linux.
Not sure if you are aware, but we do control the directory for most of what empire writes via these config properties.

[D3vil0p3r]

Yes I am aware but I cannot write directly it because yaml file cannot expand env variables in case of Linux or Windows systems. So my approach is "first, copy config.yaml to home dir ~/.empire/{server,client} and use it, then, if cannot write on directories specified on config.yaml due to Permission Error, use a fallback directory in user home folder and write this info in config.yaml.

[D3vil0p3r]

@vinnybod

1st Question

what is the difference between empire/server/data/module_source/ and empire/server/modules/ directories?

Why in empire/server/modules/ I have, for example, SpawnnProcess.yaml but in empire/server/data/module_source/ I don't have it?

These two folders are changed/written at runtime during the usage of Empire? If not, what is the reason to have

directories:
  module_source: empire/server/data/module_source/
  obfuscated_module_source: empire/server/data/obfuscated_module_source/

in server/config.yaml? If a user changes those two paths, Empire does not refer to those modules anymore (unless user copies and pastes them to its target directory) and it produces a lot of errors. If these two paths must be static, I would suggest to remove them from config.yaml. If you agree, I can do that and deal these two paths directly in the code.

2nd Question

One additional note: since you don't want users use sudo for running Empire, server.py has the following:

INVOKE_OBFS_DST_DIR_BASE = "/usr/local/share/powershell/Modules/Invoke-Obfuscation"
...
...
    # invoke obfuscation
    if os.path.exists(f"{INVOKE_OBFS_DST_DIR_BASE}"):
        shutil.rmtree(INVOKE_OBFS_DST_DIR_BASE)
    pathlib.Path(pathlib.Path(INVOKE_OBFS_SRC_DIR_BASE).parent).mkdir(
        parents=True, exist_ok=True
    )
    shutil.copytree(
        INVOKE_OBFS_SRC_DIR_BASE, INVOKE_OBFS_DST_DIR_BASE, dirs_exist_ok=True
    )

this copy will always fail due to missing permission. Still, here I would suggest to write in ~/.local/share/powershell/Modules/ so you don't need sudo.

3rd Question

When Empire is run the first time (for example on a Linux system), why is it running building/compilation by MSBuild?
Is it done for some plugins? Which one?
Where is the code triggering this building?
It is the last point I need to manage for system-wide Empire deployment before PR is ready to be reviewed.

Usually compilation must not run at runtime. Best practices suggest to do it at "building time". Is it possible you build them offline and upload the output files directly on the Empire repository so this build process code can be removed?

[vinnybod]

1st Question
what is the difference between empire/server/data/module_source/ and empire/server/modules/ directories?

The module_source directory contains source code for the modules, like the powershell code for the powershell modules. The modules contains the yaml definition for a module and optionally a python file if it needs a custom generation. There' some more info about modules here: https://bc-security.gitbook.io/empire-wiki/module-development

Why in empire/server/modules/ I have, for example, SpawnnProcess.yaml but in empire/server/data/module_source/ I don't have it?

Not every module has a 1 to 1 mapping with source code in module_source. There are modules that have the code directly in the yaml, generated dynamically, or multiple modules that map to the same source file.

If a user changes those two paths, Empire does not refer to those modules anymore (unless user copies and pastes them to its target directory) and it produces a lot of errors. If these two paths must be static, I would suggest to remove them from config.yaml

I don't recall the exact reason that these are in the config.yaml. But if I were to guess, I was moving towards a model where all the data directories that empire needs to read/write can be configurable. I'd prefer to keep them in the config for now.

2nd Question
this copy will always fail due to missing permission. Still, here I would suggest to write in ~/.local/share/powershell/Modules/ so you don't need sudo.

I am fine moving /usr/local/share/powershell/Modules/Invoke-Obfuscation to whatever directory makes the most sense in linux.

3rd Question
When Empire is run the first time (for example on a Linux system), why is it running building/compilation by MSBuild?
Is it done for some plugins? Which one?
Where is the code triggering this building?
It is the last point I need to manage for system-wide Empire deployment before PR is ready to be reviewed.
Usually compilation must not run at runtime. Best practices suggest to do it at "building time". Is it possible you build them offline and upload the output files directly on the Empire repository so this build process code can be removed?

This is coming from compiling the csharp compiler https://github.com/BC-SECURITY/Empire/blob/main/empire/server/plugins/csharpserver/csharpserver.py#L98-L100

We have actually already addressed this point in the upcoming 6.0 release which is due in a few months, so you shouldn't bother changing that. The 6.0 release will download a binary in the install script instead of compiling it at runtime.

[D3vil0p3r]

Oh ok... So... at this point, the PR is ready to be reviewed and merged. I tested on my Arch Linux and it works for both system-level and user-level deployment. without impacting your implementation with run_as_user() function.

[D3vil0p3r]

@vinnybod did you get the chance to review my changes?