Feature Request - Add support for persisting temporary access token

[adfoster-r7]

Problem

Metasploit modules may authenticate to services and retrieve temporary access tokens that allow future access to a service. For instance, cookies, JSON Web Tokens (JWT), Kerberos Tickets, OAuth, tokens etc. These values may be extracted from a compromised target, or generated with valid user/credentials.

Currently Metasploit-Credential does not offer a model for persisting these temporary access values that support future retrieval, or tracking the metadata for start/end/expiry information.

Example Values

Below is a high level summary of the types of temporary access values which can be looted from a compromised system, or retrieved from a service after authenticating.

The common themes for data requirements are:

• The token itself that is used with the service
• Time information - such as not-valid-before and not-valid-after times
• Client/Server/Service information
• Token status - valid/invalid/expired
• An associated refresh token

Cookies

Generated when interacting with an HTTP service. Often an opaque value which the server can use at a later point of time, i.e. it may be the name of a randomly generated file on disk, or a lookup value in a database.

The expiry metadata may be returned to the user in an HTTP response header, but not always:

Set-Cookie: <cookie-name>=<cookie-value>; Expires=<date>

Kerberos Tickets

Defined by https://www.rfc-editor.org/rfc/rfc4120

Kerberos tickets are used as part of authenticating with a Key Distribution Center, or extracted from memory with Kiwi etc.

The original proposal for persisting these values can be found in https://github.com/rapid7/metasploit-framework/blob/2b6cb50dc39998ab7747fd79546b31f4fff037d8/docs/metasploit-framework.wiki/Metasploit-Credential-Kerberos-Support-Proposal.md - inlined below for readability

Common Terminology:

• Service Principal Name (SPN) - Forest unique string. Associates a service to a service logon account.
  The SPN is set on a user pr computer object via the AD Schema. Generally in the form <service class>/<host><realm>:<port>/<service name>.
  A service can have multiple SPNs. Viewable on a Windows DC with setspn -q */*.
• Key Distribution Center (KDC) - Performs authentication and creates tickets

Data Requirements

Persisting the following data is required:

1. Ticket granting tickets (TGT) - Requested after using the kerberos encryption key to prove identity
2. Service Tickets (TGS) - Used to prove authenticate with a target system such as SMB etc.

There should be support for:

• Finding kerberos TGS/TGT tickets based on workspace, host (kdc), username (case insensitive), realm, sname (i.e. the kdc krbtgt/host.realm, or a service cifs/host.realm), auth time

Kerberos Tickets are associated with a Client, SPN, Realm, and have different metadata for expiry/usage times:

msf6 auxiliary(scanner/winrm/winrm_login) > klist -v
Kerberos Cache
==============
Cache[0]:
  Primary Principal: Administrator@ADF3.LOCAL
  Ccache version: 4

  Creds: 1
    Credential[0]:
      Server: krbtgt/ADF3.LOCAL@ADF3.LOCAL
      Client: Administrator@ADF3.LOCAL
      Ticket etype: 18 (AES256)
      Key: 9c66cb7de8f4d3100690771a753012eafa44a3d128342939ff9230b39aeb1713
      Subkey: false
      Ticket Length: 1090
      Ticket Flags: 0x50e10000 (FORWARDABLE, PROXIABLE, RENEWABLE, INITIAL, PRE_AUTHENT, CANONICALIZE)
      Addresses: 0
      Authdatas: 0
      Times:
        Auth time: 2022-12-13 12:57:49 +0000
        Start time: 2022-12-13 12:57:49 +0000
        End time: 2022-12-13 22:57:49 +0000
        Renew Till: 2022-12-14 12:57:49 +0000
      Ticket:
        Ticket Version Number: 5
        Realm: ADF3.LOCAL
        Server Name: krbtgt/ADF3.LOCAL
        Encrypted Ticket Part:
          Ticket etype: 18 (AES256)
          Key Version Number: 2
          Cipher:
          ...etc etc...
          ```

Scenarios

Persisting Ticket granting ticket - TGT

Context:
Can be used to request additional tickets from the KDC

Generally acquired by:

• Request from KDC using username/password/AD Realm as proof of identity
• Extracted from memory via Window’s lsa/lsass
• Forging (Golden tickets)

Example:

kdc_response = send_request(krb_enc_key)
cred = Rex::Proto::Kerberos::CredentialCache::Krb5Ccache(kdc_response)

{
  type: :tgt,
  sname: "krbtgt/realm.local",
  value: cred.to_binary_s, 
  authtime: cred.authtime.to_time,
  starttime: cred.starttime.to_time,
  endtime: cred.endtime.to_time
}

Ticket Granting Service - TGS

Context: Can be used with a real service (cifs/mssql/etc) for authentication

Generally acquired by:

• Request from KDC using a previous TGT
• Extracted from memory via Window’s lsa/lsass
• Forging (Silver tickets)

kdc_response = send_request(ticket)
cred = Rex::Proto::Kerberos::CredentialCache::Krb5Ccache(kdc_response)

{
  type: :tgs,
  sname: "arbitrary_service_class/host.realm.local:1433",
  value: cred.to_binary_s, 
  authtime: cred.authtime.to_time,
  starttime: cred.starttime.to_time,
  endtime: cred.endtime.to_time
}

Kerberos Authentication Overview

• Step 1. Request TGT
  • AS_REQ
    • Generate Kerberos Encryption key from user credentials
    • Persisted and stored for later use in a keytab file for wireshark decryption
    • Requested with sname = “krbtgt/#{realm}”
  • AS_REP
    • Stored for later usage to request future service tickets
• Step 2. Request Service Ticket
  • TGS_REQ
    • Use the TGT from Step 1
    • Specify SPN (Service principal name), i.e. cifs/host.realm.local
  • TGS_REP
    • Receive new TGS which we can use with a service
• Step 3. Interact with service
  • AP_REQ
    • Send the service ticket
  • AP_REP
    • Success/Failure information

sequenceDiagram
    participant msf as metasploit
    participant kdc as Kerberos 
    participant smb as smb

    Note over msf,kdc: 1) Request Ticket Granting Ticket - TGT
    msf->>kdc: AS_REQ<br >encKey = EncKeyFor(user, pass, realm)<br >sname = krbtgt/realm
    kdc->>msf: AS_REP<br >TGT

    Note over msf,kdc: 2) Request Service Ticket - TGS
    msf->>kdc: TGS_REQ<br>Ticket<br>spn=cifs/host.domain.local
    kdc->>msf: TGS_REP<br>TGS

    Note over msf,kdc: 3) Request Access
    msf->>smb: AP_REQ<br>Service Ticket
    smb->>msf: AP_REP

Loading

Original metasploit-credential approach

Kerberos ticket persistence originally leveraged the creds API, however it was deemed to not be viable. For instance - after running the winrm_login module this results in the following creds output:

msf6 auxiliary(scanner/winrm/winrm_login) > creds
Credentials
===========

host             origin           service            public         private                                                                                              realm       private_type  JtR Format
----             ------           -------            ------         -------                                                                                              -----       ------------  ----------
192.168.123.13   192.168.123.13   88/tcp (kerberos)  Administrator  aes256-cts-hmac-sha1-96:56c3bf6629871a4e4b8ec894f37489e823bbaecc2a0a4a5749731afa9d158e0 (TRUNCATED)  ADF3.LOCAL  Krb enc key   
192.168.123.13   192.168.123.13   88/tcp (kerberos)  Administrator  tgs:http/win10-dc3.adf3.local:ccache:BQQAAAAAAAEAAAABAAAACkFERjMuTE9DQUwAAAANQWRtaW5pc3 (TRUNCATED)  ADF3.LOCAL  Krb ticket    
192.168.123.13   192.168.123.13   88/tcp (kerberos)  Administrator  tgt:krbtgt/ADF3.LOCAL:ccache:BQQAAAAAAAEAAAABAAAACkFERjMuTE9DQUwAAAANQWRtaW5pc3RyYXRvcg (TRUNCATED)  ADF3.LOCAL  Krb ticket    
192.168.123.144  192.168.123.144  5985/tcp (http)    Administrator  p4$$w0rd                                                                                             adf3.local  Password      

msf6 auxiliary(scanner/winrm/winrm_login) > 

What we would ideally want is:

  msf6 auxiliary(scanner/winrm/winrm_login) > creds
  Credentials
  ===========
  
  host             origin           service            public         private                                                                                              realm       private_type  JtR Format
  ----             ------           -------            ------         -------                                                                                              -----       ------------  ----------
  192.168.123.13   192.168.123.13   88/tcp (kerberos)  Administrator  aes256-cts-hmac-sha1-96:56c3bf6629871a4e4b8ec894f37489e823bbaecc2a0a4a5749731afa9d158e0 (TRUNCATED)  ADF3.LOCAL  Krb enc key   
- 192.168.123.13   192.168.123.13   88/tcp (kerberos)  Administrator  tgs:http/win10-dc3.adf3.local:ccache:BQQAAAAAAAEAAAABAAAACkFERjMuTE9DQUwAAAANQWRtaW5pc3 (TRUNCATED)  ADF3.LOCAL  Krb ticket    
+ 192.168.123.13   192.168.123.13   5985/tcp (kerberos)  Administrator  tgs:http/win10-dc3.adf3.local:ccache:BQQAAAAAAAEAAAABAAAACkFERjMuTE9DQUwAAAANQWRtaW5pc3 (TRUNCATED)  ADF3.LOCAL  Krb ticket    
  192.168.123.13   192.168.123.13   88/tcp (kerberos)  Administrator  tgt:krbtgt/ADF3.LOCAL:ccache:BQQAAAAAAAEAAAABAAAACkFERjMuTE9DQUwAAAANQWRtaW5pc3RyYXRvcg (TRUNCATED)  ADF3.LOCAL  Krb ticket    
- 192.168.123.144  192.168.123.144  5985/tcp (http)    Administrator  p4$$w0rd                                                                                             adf3.local  Password       
+ 192.168.123.144  192.168.123.144  88/tcp (http)      Administrator  p4$$w0rd                                                                                             adf3.local  Password      
  
  msf6 auxiliary(scanner/winrm/winrm_login) > 

Unfortunately it's not always possible to correlate a TGS request to the service in question. i.e. in the scenario of
the user running the admin/kerberos/get_ticket module with run verbose=true rhosts=10.0.0.24 domain=mylab.local user=serviceA password=123456 action=GET_TGS spn=custom_spn/dc02.mylab.local impersonate=Administrator

Benefits:

• Tgt/Tgs information appears as part of the creds command by default
• Easy to iterate on, and we can add a separate klist command implementation later
• Easier to integrate with Pro

Shortcomings:

• Can't invalidate private information, i.e. due to tickets expiring or being rejected from a KDC etc.
• Noise in private key information
• Privates have a single data field which is assumed to be a string. It's not possible to add
  additional fields due to the current rails sti approach (single table inheritance). Which requires a workaround of using rail's serialize to store JSON blobs,
  and re-inventing active record's field validation.
• Related to the above; The default search capabilities for private creds assumes string and break on serialized json blobs
• Not everyone wants to persist/reuse keys/tickets when calling kerberos authentication, requiring hack-y "do not persist" and "do not use cache" flags
• No way to search directly via the database for SPN
• When running the creds command there's not a correlation between the SPN and the target/host that it can be used against
• Will require extra changes to Pro as part of MVP to not break the UI

Current Implementation

Metasploit currently stores Kerberos tickets in loot, and provides a simplified interface for interacting with these persisted values:

• Adding Kerberos ticket storage facade
• Add klist command

The ticket metadata is stored in the 'info' field as a serialized JSON structure (code):

• Status
• Realm
• Client
• Server

The original user/password is stored in the creds table against the target service - such as SMB. However, it should really be be stored against Kerberos:

msf6 auxiliary(scanner/winrm/winrm_login) > creds
Credentials
===========

host            origin          service          public         private   realm       private_type  JtR Format
----            ------          -------          ------         -------   -----       ------------  ----------
192.168.123.13  192.168.123.13  5985/tcp (http)  Administrator  p4$$w0rd  adf3.local  Password      

The TGT/TGS is additionally stored as loot:

msf6 auxiliary(scanner/winrm/winrm_login) > loot

Loot
====

host            service  type                 name  content                   info                                                                          path
----            -------  ----                 ----  -------                   ----                                                                          ----
192.168.123.13           mit.kerberos.ccache        application/octet-stream  realm: ADF3.LOCAL, serviceName: krbtgt/adf3.local, username: administrator    /Users/user/.msf4/loot/067f9904163fac90080d-20221202011513_default_192.168.123.13_mit.kerberos.cca_789425.bin
192.168.123.13           mit.kerberos.ccache        application/octet-stream  realm: ADF3.LOCAL, serviceName: http/dc3.adf3.local, username: administrator  /Users/user/.msf4/loot/20221207151506_default_192.168.123.13_mit.kerberos.cca_180875.bin

Benefits:

• User can see arbitrary metadata as part of the loot command

Shortcomings:

• The store_loot API can be used regardless of having a Metasploit database connected
• Kerberos tickets don't appear as part of the creds command
• Creds are stored against the wrong service, i.e. SMB instead of Kerberos

JWT

Defined by https://www.rfc-editor.org/rfc/rfc7519

JWTs contain three base64 separated values. Example:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

The three segments are:

1. Algorithm and token type, for example:

{
  "alg": "HS256",
  "typ": "JWT"
}

2. Claims, for example:

{
  "sub": "1234567890",
  "name": "John Doe",
  "iat": 1516239022
}

3. A checksum, as defined by the first segment

JWTs contain claims for tracking expiry metadata - https://www.rfc-editor.org/rfc/rfc7519#section-4.1

• "iss" (Issuer) Claim
• "sub" (Subject) Claim
• "aud" (Audience) Claim
• "exp" (Expiration Time) Claim
• "nbf" (Not Before) Claim
• "iat" (Issued At) Claim

OAuth tokens

Defined by https://www.rfc-editor.org/rfc/rfc6749

OAuth 2.0 uses the concept of Access tokens, but not does not define a specific format - but can be JWT tokens or opaque blobs.
In OAuth 2.0, Refresh Tokens also exist which can be used to request an additional token after an expiry has surpassed.

Additional context/domain knowledge

Metasploit::Credential::Core is a glue object between the various objects in metasploit-credential - here's the rendered version of the outdated credential.graffle file:

• Core - Glue object. Core credential that combines {#private}, {#public}, and/or {#realm} so that {Metasploit::Credential::Private} or {Metasploit::Credential::Public} that are gathered from a {Metasploit::Credential::Realm} are properly scoped when used
• Public - A publicly disclosed credential, i.e. a username
• Private - A private credential is any credential that should not be publicly disclosed, such as a {Metasploit::Credential::Password password}, password hash, or key file.
• Context - Was renamed to Realm
• Origin - Where the credentials came from, i.e. session/import/crackedpassword/manual/service
• Login - The use of a {#core core credential} against a {#service service}
• Realm - The realm in which a {Metasploit::Credential::Public} can be used to authenticate or from which a {Metasploit::Credential::Private} was looted.

The gem dependencies/hierarchy:

flowchart LR;
 metasploit-framework --> metasploit-concern;
 metasploit-framework --> metasploit-credential;
 metasploit-framework --> metasploit_data_models;
 metasploit-framework --> metasploit-model;

 metasploit-credential --> metasploit-concern;
 metasploit-credential --> metasploit_data_models;
 metasploit-credential --> metasploit-model;

 metasploit-concern --> metasploit-yard;
 metasploit-concern --> metasploit-erd;

 metasploit-erd --> metasploit-yard;

 metasploit_data_models --> metasploit-yard;
 metasploit_data_models --> yard-metasploit-erd;
 metasploit_data_models --> metasploit-concern;
 metasploit_data_models --> metasploit-model;

 yard-metasploit-erd --> metasploit-yard;
 yard-metasploit-erd --> metasploit-erd;

 metasploit-model --> metasploit-yard;
 metasploit-model --> metasploit-erd;

Loading

Gem overviews:

• metasploit-yard - Documentation generation
• metasploit-erd - Documentation generation
• yard-metasploit-erd - Documentation generation
• metasploit-model - Shared validators and mixins for ActiveModels in metasploit-framework and metasploit_data_models
• metasploit_data_models - Rails ORM models
• metasploit-concern - Rails/Metasploit extension library
• metasploit-credential - Rails ORM models
• metasploit-framework

[adfoster-r7]

Another consideration: We would most likely want to do a better job of storing ADCS certs in Metasploit too for future use in modules

Edit: We'll want to track arbitrary metadata on the core itself, such as the template name or oids that aren't present in the pkcs12 itself, and sid, and potentially linking it with the loot metadata filepath so folk can use it from the file system

Edit Edit: If we add metadata to the login core objects (or whatever node), we'll need to make sure it's searchable. I don't believe that's currently supported. If we use metadata store as JSONB in postgres