Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIGTOP-4023: Add ranger deb packaging support #1197

Merged
merged 2 commits into from
Oct 19, 2023

Conversation

JiaLiangC
Copy link
Contributor

@JiaLiangC JiaLiangC commented Oct 18, 2023

Description of PR

Add ranger deb packaging support
There are many installation packages for Ranegr, hence a large number of files are dynamically generated in the rules. In addition, dynamic generation is quite convenient for subsequent customization of the installation directory and package name. For deb packaging, the development of adding -PpkgSuffix and -PparentDir to most bigdata components has been completed and is currently under testing.

How was this patch tested?

manual test
image

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
  • Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/

@JiaLiangC
Copy link
Contributor Author

@sekikn Could you help review this PR?

@sekikn
Copy link
Contributor

sekikn commented Oct 19, 2023

Great work, thank you so much @JiaLiangC. I successfully built the deb packages with your PR on Ubuntu 20.04 and 22.04 (x86_64).

I have two comments:

  • When I tried to install them with the existing Puppet manifest, it failed with the following error.

    $ ./docker-hadoop.sh -d -dcp -C config_ubuntu-22.04.yaml -F docker-compose-cgroupv2.yml -G -L -k hdfs,hbase,ranger -s ranger -c 1
    
    ...
    
    Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install postgresql-jdbc' returned 100: Reading package lists...                              
    Building dependency tree...                                                                                                                                                                   
    Reading state information...                                                                                                                                                                  
    E: Unable to locate package postgresql-jdbc                                                                                                                                                   
    Error: /Stage[main]/Ranger::Prerequisites/Package[postgresql-jdbc]/ensure: change from 'purged' to 'latest' failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=
    --force-confold install postgresql-jdbc' returned 100: Reading package lists...                                                                                                               
    Building dependency tree...                                                                                                                                                                   
    Reading state information...                                                                                                                                                                  
    E: Unable to locate package postgresql-jdbc                                                                                                                                                   
    Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install postgresql-server' returned 100: Reading package lists...                            
    Building dependency tree...                                                                                                                                                                   
    Reading state information...                                                                                                                                                                  
    E: Unable to locate package postgresql-server                                                                                                                                                 
    Error: /Stage[main]/Ranger::Prerequisites/Package[postgresql-server]/ensure: change from 'purged' to 'latest' failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options:
    :=--force-confold install postgresql-server' returned 100: Reading package lists...                                                                                                           
    Building dependency tree...                                                                                                                                                                   
    Reading state information...                                                                                                                                                                  
    E: Unable to locate package postgresql-server                                                                                                                                                 
    

    The current Puppet manifest is tested only on RPM and assumes RPM package names for PostgreSQL. But there's no "postgresql-jdbc" and "postgresql-server" packages on deb-based system.
    I suppose adding dependencies on PostgreSQL in Ranger's deb control file and removing the package declaration for Postgres from the Puppet manifest will solve the problem, as the Ambari component already did.
    Would you fix this problem in this issue? Or you can file it as another issue and address it separately. Either is fine.

  • (minor nit) There are other component names (hbase and hive) in the copyright and postinst files. Would you fix them?

fix copyright and ranger.postinst.tpl file typo
@JiaLiangC
Copy link
Contributor Author

@sekikn
Thank you very much for helping with the review, the modifications have been completed. Here is the related Puppet PR #1199.

@sekikn
Copy link
Contributor

sekikn commented Oct 19, 2023

+1, I made sure the build succeeded on Debian 11 and Ubuntu 22.04 with this PR, and installing ranger-admin installs postgresql as its dependency.
Thank you for your contribution @JiaLiangC.

@sekikn sekikn merged commit c4b34b2 into apache:master Oct 19, 2023
Nazarii-Melnyk pushed a commit to rework-space-com/bigtop that referenced this pull request Sep 20, 2024
* BIGTOP-4023: Add ranger deb packaging support

* add postgresql dependency
fix copyright and ranger.postinst.tpl file typo

---------

Co-authored-by: jialiang <jialiangcaimd@gmail.com>
(cherry picked from commit c4b34b2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants