Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job db #30

Merged
merged 62 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
c38a13b
Initial failing browser db test
SvenMarcus May 2, 2023
ca7f4df
Passes first DB test. Needs refactoring!
SvenMarcus May 3, 2023
5542972
Begin implementing repository
SvenMarcus May 4, 2023
a525a1d
Initial failing browser db test
SvenMarcus May 2, 2023
3f0e307
Passes first DB test. Needs refactoring!
SvenMarcus May 3, 2023
c908ad8
Begin implementing repository
SvenMarcus May 4, 2023
aaf6fec
Refactor DB test. Make repository setup async
SvenMarcus May 5, 2023
0801e99
Merge branch 'feature/browser-db' of https://github.com/SvenMarcus/oc…
SvenMarcus May 8, 2023
4e625e8
Work on test setup
SvenMarcus May 11, 2023
91ca4df
Refactor fixtures. Begin refactor towards running browser
SvenMarcus May 16, 2023
6a321eb
Factory responsible for starting browsers.
SvenMarcus May 19, 2023
9e57fee
Remove RedirectMap from workspace endpoint. Restoring factory control…
SvenMarcus May 23, 2023
1406a15
Use FastAPI dependency injection
SvenMarcus Jun 5, 2023
a0877cf
Update docker-compose.yml and init.sh for MongoDB
SvenMarcus Jun 5, 2023
25683e0
Working Docker Browsers. Subprocesses are crashing
SvenMarcus Jun 6, 2023
9124bf3
Working subprocess browsers
SvenMarcus Jun 7, 2023
fe176f1
Try ports instead of taking from set
SvenMarcus Jun 12, 2023
59ae366
Mark launch tests as integration
SvenMarcus Jun 12, 2023
f7cdf42
Add simple test to ping browser. Remove unused Port class
SvenMarcus Jun 13, 2023
f16cff5
Build browse-ocrd docker image in CI
SvenMarcus Jun 13, 2023
d1a0167
Introduce first method for repository
SvenMarcus Jun 13, 2023
fd4767b
Fix mypy failure
SvenMarcus Jun 13, 2023
94a5966
Clean unreachable browsers on startup. Refactor workspace endpoint
SvenMarcus Jun 14, 2023
546aecd
Catch exception when killing browser fails
SvenMarcus Jun 19, 2023
98d03ce
Pass SubProcessBrowser Tests. Refactor towards common port binding fu…
SvenMarcus Jun 21, 2023
a8bf2b8
Better, but still insufficient, handling of native browser launches
SvenMarcus Jun 23, 2023
4efba42
Fix duplicate window bug
SvenMarcus Jul 14, 2023
037633c
Fix the tests
SvenMarcus Jul 14, 2023
de3c815
Passes mypy
SvenMarcus Jul 14, 2023
18519c0
Introduce Fixture facade
SvenMarcus Jul 18, 2023
5554c85
Refactor existing tests for Fixture API
SvenMarcus Jul 18, 2023
9246c61
Refactor existing tests for Fixture API
SvenMarcus Jul 18, 2023
f6babea
Remove unnecessary markers
SvenMarcus Jul 19, 2023
50acaa0
ocrd_browser.workspace_dir must be 'ocr-d' subdirectory only
bertsky Jul 3, 2023
e52ee33
database dependency: condition started
bertsky Jul 3, 2023
22176ed
no explicit container name (rely on compose aliasing DNS services)
bertsky Jul 3, 2023
a61b034
db express: control exposed port via MONITOR_PORT_DBE=8081
bertsky Jul 3, 2023
043b771
also use MongoDB to manage OCR-D jobs…
bertsky Jul 6, 2023
7fe8754
jobs: fix typo
bertsky Jul 9, 2023
e2597e9
kill active job by PID via Manager webserver
bertsky Jul 9, 2023
7028d6c
Begin reworks for job db
SvenMarcus Jul 20, 2023
7d3eb92
Finish restructure for job db
SvenMarcus Jul 23, 2023
c1e7cfe
Remove unused imports
SvenMarcus Jul 24, 2023
5d0edbc
Prefix DB env variables with MONITOR
SvenMarcus Jul 24, 2023
27877d5
Rename missed variables
SvenMarcus Jul 24, 2023
9d766bd
Change settings to match new env names
SvenMarcus Jul 24, 2023
8272758
Fix connection string in prod env
SvenMarcus Jul 24, 2023
54126b8
Fix long line in logs
SvenMarcus Jul 25, 2023
8538894
Add devcontainer config
SvenMarcus Aug 1, 2023
2d74cca
Fix monitor db connection string, rename mongodb express service
markusweigelt Aug 4, 2023
8467cf2
using setting in pydantic v2
markusweigelt Aug 7, 2023
f84999c
Add pydantic settings module
markusweigelt Aug 7, 2023
c084c60
Add pydantic settings module and update dependencies to run pydantic
markusweigelt Aug 7, 2023
115dbc5
Migrate settings to pydantic v2
SvenMarcus Aug 8, 2023
82786b0
Add ocrd browser
markusweigelt Aug 9, 2023
e44af1d
Add missing newlines
markusweigelt Aug 9, 2023
604bfec
Install necessary deps in test ci
markusweigelt Aug 9, 2023
3dca921
Install necessary deb in test ci
markusweigelt Aug 9, 2023
8a9a98a
Install and using pdm
markusweigelt Aug 9, 2023
5982a4c
Install necessary deps
markusweigelt Aug 9, 2023
a9ae2d3
Install using dev group
markusweigelt Aug 9, 2023
64ebfaf
Change installation of deps in test ci
markusweigelt Aug 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
FROM mcr.microsoft.com/vscode/devcontainers/python:3

# [Option] Install zsh
ARG INSTALL_ZSH="true"
# [Option] Upgrade OS packages to their latest versions
ARG UPGRADE_PACKAGES="false"
# [Option] Enable non-root Docker access in container
ARG ENABLE_NONROOT_DOCKER="true"
# [Option] Use the OSS Moby Engine instead of the licensed Docker Engine
ARG USE_MOBY="true"
# [Option] Engine/CLI Version
ARG DOCKER_VERSION="latest"

# Enable new "BUILDKIT" mode for Docker CLI
ENV DOCKER_BUILDKIT=1

# Install needed packages and setup non-root user. Use a separate RUN statement to add your
# own dependencies. A user of "automatic" attempts to reuse an user ID if one already exists.
ARG USERNAME=automatic
ARG USER_UID=1000
ARG USER_GID=$USER_UID
COPY library-scripts/*.sh /tmp/library-scripts/
RUN apt-get update \
&& /bin/bash /tmp/library-scripts/common-debian.sh "${INSTALL_ZSH}" "${USERNAME}" "${USER_UID}" "${USER_GID}" "${UPGRADE_PACKAGES}" "true" "true" \
# Use Docker script from script library to set things up
&& /bin/bash /tmp/library-scripts/docker-in-docker-debian.sh "${ENABLE_NONROOT_DOCKER}" "${USERNAME}" "${USE_MOBY}" "${DOCKER_VERSION}" \
# Clean up
&& apt-get autoremove -y && apt-get clean -y && rm -rf /var/lib/apt/lists/* /tmp/library-scripts/

VOLUME [ "/var/lib/docker" ]

# Setting the ENTRYPOINT to docker-init.sh will start up the Docker Engine
# inside the container "overrideCommand": false is set in devcontainer.json.
# The script will also execute CMD if you need to alter startup behaviors.
ENTRYPOINT [ "/usr/local/share/docker-init.sh" ]
CMD [ "sleep", "infinity" ]


RUN apt-get update \
&& apt-get install -y --no-install-recommends libcairo2-dev libgtk-3-bin libgtk-3-dev libglib2.0-dev libgtksourceview-3.0-dev libgirepository1.0-dev gir1.2-webkit2-4.0 pkg-config cmake \
&& pip3 install -U setuptools \
&& pip3 install browse-ocrd

RUN pipx install pdm
23 changes: 23 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"build": {
"dockerfile": "Dockerfile"
},
"customizations": {
"vscode": {
"extensions": [
"ms-python.black-formatter",
"usernamehw.errorlens",
"ms-python.python",
"ms-python.vscode-pylance",
"vscodevim.vim",
"charliermarsh.ruff"
]
}
},
"remoteEnv": {
"LOCAL_WORKSPACE_FOLDER": "${localWorkspaceFolder}"
},
"runArgs": ["--init", "--privileged"],
"mounts": ["source=dind-var-lib-docker,target=/var/lib/docker,type=volume"],
"overrideCommand": false
}
Loading