Get access to CertBot User-Agent field

[positiwise]

Hello, Can you suggest best way to access Certbot User-agent and user-agent-comments fields that are passed from certbot to access from CAHandler?

[grindsa]

Hi, there is no easy way to do this without changing the code.

Can you describe the use-case behind your request a bid more in detail? Further are you only interested in the user-agent and comments fields or some more?

[positiwise]

Thanks for quick reply.

We are writing custom CAHandler for our clients using acme2certifier. The use case is we need some custom data to be passed from certbot and user-agent, user-agent-comments fields seemed most "standard" way to pass data to acmeserver. We are creating certbot+ ie a layer on top of certbot to ease client-side as well.

Further to this, I am running as wsgi and able to see environ having values correctly with "HTTP_USER_AGENT" that is passed from certbot but nothing for user-agent-comments.

So lets say we are able to get these value in "application(environ,...)", what is the quickest way to make sure CAHandler can access this value? I want to minimize changes to ACME2Certifier as little as possible to ease future maintenance.

[grindsa]

Thank you for the explanation. The idea is crazy but in a way that I like it.

Anyway, passing additional date to the CA-handler would be a breaking change and require modification. on each CAhandler. Given the fact that I do not have any clue how many customized ca.handlers are in the field I would like to avoid this.

How about the following approach (requires some but reasonable changes in a2c):

• we make it configurable, meaning you can specify the http-header attributes to be parsed in acme_srv.cfg
• we expect these attributes to be set / header to be send along with the CSR (meaning as part of the order-finalization message).
• we store attributes and values as serialized json in the certificate table of a2c database
• the CAhandler can lookup these attributes and values based on the csr he gets submitted and use it accordingly to his needs

This should fulfill your ask but with the flexibility to use any header attribute to send meta-information.

Thoughts?

[positiwise]

This sounds like a great solution, thank you. For now, I will do local modifications, and will adapt official version, whenever it's available.

[grindsa]

ok As i said. Effort is reasonable. So you should get some code to play with soon...

[grindsa]

I think the first code for testing will be ready beginning of next week. Will you install manually, or do you prefer a wsgi docker image, rpm or deb?

[positiwise]

Thanks for quick turnaround, i prefer manually as we are in dev phase still.

[grindsa]

Ok here we go. First code is ready. I created a separate branch for testing. Below some instructions:

This version requires a database-update. Please create a backup of acme_srv/acme_srv.db and run the update script tools/db_update.py

You need to modify the acme_srv/acme_srv.cfg and add the header attributes you would like to dump to database:

[Order]
header_info_list: ["HTTP_USER_AGENT", "CONTENT_TYPE", "REMOTE_ADDR"]

The headers will be added into the header_info column of the certificates table. (you and browse acme_srv.db with any SQLite browser).

You can load the header information with the header_info_get() function from helper.py as serialized json. The skeleton_handler contains an example.

Output from the above example would be:

2023-11-03 16:52:14 - acme2certifier - WARNING - {"HTTP_USER_AGENT": "CertbotACMEClient/1.21.0 (certbot; Ubuntu 22.04.3 LTS) Authenticator/standalone Installer/None (certonly; flags: ) Py/3.10.12", "CONTENT_TYPE": "application/jose+json", "REMOTE_ADDR": "192.168.14.131"}

Please give it a try and let me know if it works for you. Feedback is always welcome...

[positiwise]

Thank you, let me check and come back to you.

[positiwise]

Hello is there any update to db_handler.py? I don't see it in this branch? My sqlitedb is testing only, so if there's any other issue, please send me acme.db blank.

This is the error I am facing currently on this branch :

Exception in thread Thread-6 (_enroll_and_store):

[Mon Nov 06 16:28:54.845024 2023] [wsgi:error] [pid 14867:tid 140462958147136] Traceback (most recent call last):

[Mon Nov 06 16:28:54.845029 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner

[Mon Nov 06 16:28:54.845181 2023] [wsgi:error] [pid 14867:tid 140462958147136] self.run()

[Mon Nov 06 16:28:54.845185 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/var/www/acme2certifier/acme_srv/threadwithreturnvalue.py", line 17, in run

[Mon Nov 06 16:28:54.845208 2023] [wsgi:error] [pid 14867:tid 140462958147136] self._return = self._target(*self._args, **self._kwargs)

[Mon Nov 06 16:28:54.845211 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/var/www/acme2certifier/acme_srv/certificate.py", line 392, in _enroll_and_store

[Mon Nov 06 16:28:54.845270 2023] [wsgi:error] [pid 14867:tid 140462958147136] (error, certificate, certificate_raw, poll_identifier) = self._enroll(csr, ca_handler)

[Mon Nov 06 16:28:54.845278 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/var/www/acme2certifier/acme_srv/certificate.py", line 281, in _enroll

[Mon Nov 06 16:28:54.845320 2023] [wsgi:error] [pid 14867:tid 140462958147136] (error, certificate, certificate_raw, poll_identifier) = ca_handler.enroll(csr)

[Mon Nov 06 16:28:54.845323 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/var/www/acme2certifier/examples/ca_handler/acme-tss.py", line 62, in enroll

[Mon Nov 06 16:28:54.845347 2023] [wsgi:error] [pid 14867:tid 140462958147136] qset = header_info_get(self.logger, csr=csr)

[Mon Nov 06 16:28:54.845350 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/var/www/acme2certifier/acme_srv/helper.py", line 538, in header_info_get

[Mon Nov 06 16:28:54.845416 2023] [wsgi:error] [pid 14867:tid 140462958147136] result = dbstore.certificates_search('csr', csr, vlist)

[Mon Nov 06 16:28:54.845419 2023] [wsgi:error] [pid 14867:tid 140462958147136] File "/var/www/acme2certifier/acme_srv/db_handler.py", line 972, in certificates_search

[Mon Nov 06 16:28:54.845565 2023] [wsgi:error] [pid 14867:tid 140462958147136] result[ele] = lookup[ele]

[positiwise]

I think we will need updated db_handler.py so that we could have access to methods for this branch that saves and retrives header_info

[grindsa]

Correct! My above instructions are incomplete. In addition, you need to

• copy example/db_handler/wsgi_handler.,py to acme_srv/db_handler.py
• replace your ./acme2certifier_wsgi.py with the version stored in examples directory

Sorry for causing confusion...

Please do not forget to run the tools/db_update.py script afterwards

[positiwise]

Thank you for your time on this. It really works!

[grindsa]

Glad to hear this. I will merge the changes into devel-branch and release it in the upcoming days.

[grindsa]

Feature has been included in a2c v0.30. Thus I am closing this discussion.