Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue-268: Changes needed to allow RHEL based builds via Islandora-playbook #268 #273

Merged
merged 26 commits into from
Jan 19, 2024
Merged

Issue-268: Changes needed to allow RHEL based builds via Islandora-playbook #268 #273

merged 26 commits into from
Jan 19, 2024

Conversation

bdgregg
Copy link
Contributor

@bdgregg bdgregg commented Jan 16, 2024

GitHub Issue: #268

  • Other Relevant Links (Google Groups discussion, related pull requests,
    Release pull requests, etc.)

What does this Pull Request do?

This pull request largely enables the ansible build of RedHat based systems to go
along with already existing Debian based system build process. There are a few other
changes that we felt could also be included. A brief listing of these changes is included below.

What's new?

A in-depth description of the changes made by this PR. Technical details and
possible side effects.

  • Changes x feature to such that y
  • Added x
  • Removed y
  • Does this change require documentation to be updated?
  • Does this change add any new dependencies?
  • Does this change require any other modifications to be made to the repository
    (i.e. Regeneration activity, etc.)?
  • Could this change impact execution of existing code?

The primary focus of this pull request is to allow for building RedHat based systems (RedHat) the same way that Debian based systems are built using the Islandora-playbook. As such the majority of the changes are tied into accommodating RedHat based approaches to the installation of the same components used by Debian or using an equivalent package repositories for the same components. A number of changes relate to changing the default locations of variables used for RedHat instead of Debian. There are a few other changes that we identified that we felt would fit better for both RedHat and Debian.

  • No documentation changes are required. Following the existing Islandora-playbook documentation should be sufficient.
  • No new dependencies besides the inclusion of the rpmfusion repo for RHEL based systems are included.
  • No changes are required to be made the repository (i.e. Regeneration activity, etc.).
  • These changes should not impact execution of existing code other than accommodating RedHat based systems.

See below for a brief description of the list of changes.

Primary Changes:

Files/Roles that were updated:
requirements.yml - There were a few changes in versions of geerlingguy modules that addressed issues when running these modules under RHEL.

Islandora-Devops.activemq - Adjusted how ActiveMQ is downloaded to add a timeout value as we were experiencing an issue with this. Also instead of using 'mv', we opted for creating a symlink to the extracted content. This allowed us to retain the version information from the original file extraction in the folder name but also providing the activemq folder to be used generally.

Islandora-Devops.alpaca - Adjusted obtaining the JAVA_HOME environment variable for RHEL.

Islandora-Devops.blazegraph - Adjusted the location of variables based upon OS based vars instead of hardcoded in defaults/main.yml. Created tasks/config-RedHat.yml and tasks/define-home.yml to pull in the OS based variable. Also adjusted the install.yml to be renamed install-Debian.yml and then created install-RedHat.yml. Created vars/Debian.yml and vars/RedHat.yml to store the OS based variable. Adjusted tasks/main.yml to reflect the above changes.

Islandora-Devops.canaloupe - Adjusted the location of variables based upon OS based vars instead of hardcoded in defaults/main.yml. Moved tasks/install.yml to tasks/install-Debian.yml and created tasks/install-RedHat.yml. Created tasks/os-vars.yml to pull in OS based vars and adjusted tasks/main.yml to reflect the above changes.

Islandora-Devops.crayfish - Added the rpmfusion repository for RedHat 8 ffmpeg instead of the nux repo and updated the when clauses to accommodate the change based upon OS flavor and version.

Islandora-Devops.fcrepo-syn - Adjusted the location of variables based upon OS based vars instead of hardcoded in defaults/main.yml similar to other previously mentioned modules above.

Islandora-Devops.fcrepo - Adjusted the python dependencies to only install either PostgreSQL or MySQL/MariaDB python modules. RedHat only allows for one or the other installed due to a conflict. As the database variant is chosen elsewhere (see variable islandora_db) it seemed fitting that one or the other and not both are needed at the same time.

Islandora-Devops.matomo - Adjusted the SQL to use a different collation when used with RedHat/MySQL/MariaDB. Collation 'utf8mb4_0900_ai_ci' is not a recognized collation and using the collation of utf8mb4_unicode_ci resolved the error: "Unknown collation: ‘utf8mb4_0900_ai_ci’ " when importing the database.

Islandora-Devops.tomcat - Adjusted how Tomcat is setup in RedHat and also pulling default variables from OS specific vars instead of the defaults/main.yml file similar to those mentioned above.

solr.yml - Updated the version of Java from 1.8.0 to Java 11 for RedHat. Included installation of zip package for RedHat.

tomcat.yml - Updated the version of Java from 1.8.0 to Java 11 for RedHat.

bootstrap.yml - Exchanged the ansible execution of the command "setenforce permissive" to using the ansible selinux module to perform the same function.

crayfish.yml - Changed to calling roles using "role:" instead of "name:" for bit of clarity to distinguish the calling of a role from other "name:" instances and to match other already existing "role:" conventions in the code.

post-install.yml - Add when clause to run tasks only for Debian based systems. This excludes the tasks from running against RedHat based systems where folders differ between Debian and RedHat.

vars/RedHat.yml - Added selinux_permissive variable to allow bootstrap.yml to set selinux settings for RedHat.

Other Changes:

.gitignore - Excluded all other folders under the inventory folder besides vagrant from being captured by git.

ansible.cfg - Included color stanza for "verbose = bright blue" so as to have verbose ansible messages show with better contrast against a black terminal background.

How should this be tested?

A description of what steps someone could take to:

  • Reproduce the problem you are fixing (if applicable)
  • Test that the Pull Request does what is intended.
  • Please be as detailed as possible.
  • Good testing instructions help get your PR completed faster.

Testing of this pull request should follow the existing Islandora-playbook documentation for a newly built RedHat based system. Our RedHat testing was done with a RedHat 8.9 (Ootpa) x86_64 VM installation. Additionally this pull request should continue to work with Debian based systems. To confirm the Debian setup was still functional our Debian testing was done with Ubuntu 22.04.03 LTS (Jammy Jellyfish). These two test systems were VM's in our current VMware vSphere 7.0 environment. For our Ansible host we used our already existing ansible system (RedHat 7 with the standard ansible 2.9.27-1 package from EPEL) to invoke the playbook to our test systems.

From our ansible playbook directory we followed the steps outlined in "Deploying to a remote environment" at https://islandora.github.io/documentation/installation/playbook/#deploying-to-a-remote-environment

Additional Notes:

Any additional information that you think would be helpful when reviewing this
PR.

NOTE: Although the resulting RedHat system seems to build and work properly, we are experiencing an issue with derivative creation for some images resulting in not having a FITS media file created. We are unable to ascertain what the root cause of this is and are asking that someone that is more familiar with the derivative generation components to investigate to see if we missed something in the configuration or settings wise.

Interested parties

Tag (@ mention) interested parties or, if unsure, @Islandora-Devops/committers

bdgregg added 26 commits October 4, 2023 16:01
@bdgregg
Islandora-Devops#268 Fixing rhel installation of zip.
4d2de22
@bdgregg
Islandora-Devops#268 Redhat needs tomcat9_config_dir created.
9b25647
@bdgregg
Islandora-Devops#268 Adjust islandora_build_box to islandora_build_ba…
9769b56 
…se_box and change name to role in playbook to keep in line with other playbooks.
@bdgregg
Islandora-Devops#268 Use Ansible selinux module instead.
f558977
@bdgregg
Islandora-Devops#268 Define selinux_permissive before using set_fact …
4014116 
…in bootstrap.yml.
@bdgregg
Islandora-Devops#268 python3-pgsopg2 conflicts with python3-mysqlclie…
6ff5b9a 
…nt - need to split the installation of packages out depending upon islandora_db variable as well.
@bdgregg
Islandora-Devops#268 refactor wait after SOLR restart.
d057f1b
@bdgregg
Islandora-Devops#268 fix for wait_for pause after SOLR restart.
b29c653
@bdgregg
Islandora-Devops#268 Resolves collation error when importing matomo.s…
a881c99 
…ql file into MariaDB. Changed colation from utf8mb4_0900_ai_ci to utf8mb4_unicode_ci.
@bdgregg
Draft changes to support RHEL.
144781d
@bdgregg
Draft file additions to support RHEL.
fef8d12
@bdgregg
Cleaned up commented out code.
d8addde
@bdgregg
Updated .gitignore.
b90e9c4
@bdgregg
A bit of general cleanup.
534ff2a
@bdgregg
Additional Cleanup.
0308423
@bdgregg
Additional Cleanup - tomcat9.service file for Debian.
3f079ac
@bdgregg
Additional Cleanup - blazegraph vars.
d1cee94
@bdgregg
Additional Cleanup - cantaloupe tasks.
74ededa
@bdgregg
Additional Cleanup - fcrepo-syn tasks/config.Debian.
8822cec
@bdgregg
Additional Cleanup - fcrepo-syn remove debug from main.yml.
2b13882
@bdgregg
Additional Cleanup - fcrepo remove trailing line from tasks/config.yml.
9adeb3e
@bdgregg
Additional Cleanup - alpaca minor changes to templates/startup.sh.
ae9c759
@bdgregg
Additional Cleanup - alpaca whitespace changes to templates/startup.sh.
1079b05
@bdgregg
Additional Cleanup - blazegraph remove commented code tasks/main.yml.
b635c43
@bdgregg
Additional Cleanup - remove commented code tomcat.yml.
dc29eae
@bdgregg
Additional Cleanup - remove trailing space.
ebc6cff
@seth-shaw-asu
Copy link
Contributor

So, just for clarity, this is intended for an existing RedHat box, not for building a Vagrant/VirtualBox RedHat instance, correct?

This worked as expected of a default debian install. I also tried to change the ISLANDORA_DISTRO to generic/rhel7 but that failed out when it couldn't find python3 on the box.

@bdgregg
Copy link
Contributor Author

bdgregg commented Jan 18, 2024 via email

@seth-shaw-asu
Copy link
Contributor

Well, I couldn't get the generic RH or RockyLinux vagrant base boxes to work, but since it doesn't break the official debian version we use in the Vagrant file, I'm going to approve this.

@seth-shaw-asu seth-shaw-asu merged commit d4002ac into Islandora-Devops:dev Jan 19, 2024
@bdgregg
Copy link
Contributor Author

bdgregg commented Jan 19, 2024 via email

@bdgregg
Copy link
Contributor Author

bdgregg commented Feb 14, 2024 via email

@seth-shaw-asu
Copy link
Contributor

Simply setting ISLANDORA_DISTRO to generic/rocky8 with a fresh clone of the playbook produced the following error:

TASK [include OS specific variables] *******************************************
Tuesday 20 February 2024  16:20:40 -0800 (0:00:00.932)       0:00:00.982 ****** 
fatal: [default]: FAILED! => {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": "Could not find or access 'vars/Rocky.yml'\nSearched in:\n\t/home/sshaw28/islandora-playbook-cent/vars/vars/Rocky.yml\n\t/home/sshaw28/islandora-playbook-cent/vars/Rocky.yml\n\t/home/sshaw28/islandora-playbook-cent/vars/vars/Rocky.yml\n\t/home/sshaw28/islandora-playbook-cent/vars/Rocky.yml on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}

Is there another variable I needed to set as well?

@bdgregg
Copy link
Contributor Author

bdgregg commented Feb 21, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seth-shaw-asu seth-shaw-asu seth-shaw-asu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bdgregg @seth-shaw-asu