Using obs_scm and extracting files in the same source _service #223

trevorsandy · 2018-02-19T15:53:19Z

Problem

I've tried several configurations, read the online documentation and reviewed the issues and pull requests but I'm not able to extract files and use obs_scm in the same _service.

Objective

I would like to produce a <package>.obsinfo file in order to extract the Git sha. I use the sha as part of the application version line displayed in the application's 'about' dialogue.

Current configuraiton

I use the OBS Reference Server.

Currently, I use tar_scm and extract_file in source my _service. I update my version string via Git's pre-commit function. The version_info file is consumed during OBS' source compile to set the application's version displayed in the about dialogue. However, as you can imagine, the sha is always one behind as the pre-commit will use the current commit details versus the one it is about the confirm.

To build my Debian distros and AppImage, I extract the required source files using the extract_file source service.

Desired behaviour

I would like to capture the git sha without disturbing the current obs_scm's version/version_format/filename parameters as my rules (Debian) and spec file build logic uses the current values and I would have to revise these components to ensure successful build.

obs_scm will produce the <package>.obsinfo file which I can use to capture the current commit sha at build time.

Problem

I cannot find the required configuration to use both obs_scm and extract_file in the same source _service file. I read in the online documentation that obs_scm has an extract parameter but I can't find any reference describing how to use this parameter and if it supports placing files in folders as is required for Debian builds.

Expected behaviour

I'm hopeful someone with more knowledge of OBS can help be better understand what is possible and whether my desired behavior is achievable with the configuration I have.

Here is an example of my current _service file:

<services>
  <service name="tar_scm">
      <param name="url">git://github.com/user/myApp</param>
      <param name="scm">git</param>
      <param name="exclude">windows/tools</param>
      <param name="version">git-master</param>
      <param name="versionformat">git</param>
      <param name="filename">myapp</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/compat</param>
      <param name="outfilename">debian.compat</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/copyright</param>
      <param name="outfilename">debian.copyright</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/postrm</param>
      <param name="outfilename">debian.postrm</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/postinst</param>
      <param name="outfilename">debian.postinst</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/rules</param>
      <param name="outfilename">debian.rules</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/changelog</param>
      <param name="outfilename">debian.changelog</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/control</param>
      <param name="outfilename">debian.control</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/source/format</param>
      <param name="outfilename">debian.source.format</param>
  </service>
  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/myApp/images/myApp.xpm
                          */builds/linux/obs/myApp-rpmlintrc
                          */builds/linux/obs/appimage.yml
                          */builds/linux/obs/debian/myApp.dsc
                          */builds/linux/obs/myApp.spec
                          */builds/linux/obs/PKGBUILD
      </param>
  </service>
  <service name="recompress">
    <param name="file">*.tar</param>
    <param name="compression">gz</param>
  </service>
</services>

The text was updated successfully, but these errors were encountered:

adrianschroeter · 2018-02-19T15:56:46Z

On Montag, 19. Februar 2018, 16:53:20 CET wrote Trevor SANDY:

#### Problem I've tried several configurations, read the online documentation and reviewed the issues and pull requests but I'm not able to extract files and use **obs_scm** in the same _service. #### Objective I would like to produce an `<package>.obsinfo` file in order to extract the Git sha. I use the sha as part of the application version line displayed in the application's 'about' dialogue. #### Current configuraiton I use the OBS Reference Server. Currently, I use `tar_scm` and `extract_file` in source my _service. I update update my version string via the Git's pre-commit function. The version_info file is consumed during OBS compile to set the application's version displayed in the about dialogue. However, as you can imagine, the sha is always one behind as the pre-commit will use the current commit details versus the one it is about the confirm. To build my Debian distros and AppImage, I extract the required source files using the `extract_file` source service. #### Desired behaviour I would like to capture the git sha without disturbing the current **obs_scm**'s version/version_format/filename parameters as my rules (Debian) and spec file build logic uses the current values and I would have to revise these components to ensure successful build. **obs_scm** will produce the `<package>.obsinfo` file which I can use to capture the current commit sha at build time. #### Problem I cannot find the required configuration to use both **obs_scm** and **extract_file** in the same source _service file. I read in the online documentation that `obs_scm` has and `extract` parameter but I can't find any reference describing how to use this parameter and if it supports placing files in folders as is required for Debian builds

You don't need extract_file usually, since you can tell obs_scm already to extract via the "extract" parameter. Have a look here for an example: https://build.opensuse.org/package/view_file/OBS:Server:Unstable/build/_service?expand=1 Is that helpful?

…

-- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany

trevorsandy · 2018-02-19T19:27:14Z

Wow! This is precisely what I was looking for. Many thanks.

...I read in the online documentation that obs_scm has an extract parameter but I can't find any reference describing how to use this parameter and if it supports placing files in folders as is required for Debian builds

Indeed, I did read in the documentation that obs_scm has an extract parameter. I just couldn't find a reference or example.

I'll add the example you provided here so anyone looking for the same solution will have a better chance of finding it than I did.

<services>
  <service name="obs_scm">
    <param name="versionformat">%ad</param>
    <param name="url">git://github.com/openSUSE/obs-build.git</param>
    <param name="scm">git</param>
    <param name="extract">dist/build.changes</param>
    <param name="extract">dist/build.spec</param>
    <param name="extract">dist/build.dsc</param>
    <param name="extract">dist/debian.changelog</param>
    <param name="extract">dist/debian.compat</param>
    <param name="extract">dist/debian.control</param>
    <param name="extract">dist/debian.rules</param>
    <param name="extract">dist/PKGBUILD</param>
  </service>
  <service name="tar" mode="buildtime"/>
  <service name="recompress" mode="buildtime">
    <param name="compression">gz</param>
    <param name="file">*.tar</param>
  </service>
  <service name="set_version" mode="buildtime"/>
</services>

Cheers,

trevorsandy · 2018-02-19T19:52:28Z

One more question...
How would I transform the line <param name="files">*/builds/linux/obs/debian/changelog</param> in:

  <service name="extract_file">
      <param name="archive">*.tar</param>
      <param name="files">*/builds/linux/obs/debian/changelog</param>
      <param name="outfilename">debian.changelog</param>
  </service>

to <param name="extract">dist/debian.changelog</param> in:

<services>
  <service name="obs_scm">
    <param name="versionformat">%ad</param>
    <param name="url">git://github.com/openSUSE/obs-build.git</param>
    <param name="scm">git</param>
    <param name="extract">dist/debian.changelog</param>
</services>

Would something like <param name="extract">dist/builds/linux/obs/debian.changelog</param> work ?

trevorsandy · 2018-02-19T21:34:10Z

Aside from the exceptions below, with this _service, all the images are unresolvable with either nothing provides obs-service-tar, nothing provides obs-service-recompress, or both returned messages...

<services>
  <service name="obs_scm">
      <param name="url">git://github.com/trevorsandy/myApp</param>
      <param name="scm">git</param>
      <param name="exclude">windows/tools</param>
      <param name="version">git-master</param>
      <param name="versionformat">git</param>
      <param name="filename">myApp</param>
      <param name="extract">builds/linux/obs/debian/compat</param>
      <param name="extract">builds/linux/obs/debian/copyright</param>
      <param name="extract">builds/linux/obs/debian/postrm</param>
      <param name="extract">builds/linux/obs/debian/postinst</param>
      <param name="extract">builds/linux/obs/debian/rules</param>
      <param name="extract">builds/linux/obs/debian/source/format</param>    
      <param name="extract">builds/linux/obs/alldeps/debian/changelog</param>
      <param name="extract">builds/linux/obs/alldeps/debian/control</param>
      <param name="extract">builds/linux/obs/alldeps/debian/myApp.dsc</param>
      <param name="extract">builds/linux/obs/alldeps/appimage.yml</param>
      <param name="extract">builds/linux/obs/alldeps/myApp.spec</param>
      <param name="extract">builds/linux/obs/myApp-rpmlintrc</param>    
      <param name="extract">builds/linux/obs/alldeps/PKGBUILD</param>
      <param name="extract">mainApp/images/myapp.xpm</param>    
  </service>
  <service name="tar" mode="buildtime"/>
  <service name="recompress" mode="buildtime">
    <param name="compression">gz</param>
    <param name="file">*.tar</param>
  </service>
</services>

The exceptions (successful compiles) are...

AppImage
openSUSE_Factory
openSUSE_Leap_42.2 [x86_64 only]
openSUSE_Tumbleweek

It seems like the failed images does not like the mode="buildtime" flag ?

Moreover, if a source file resides in a subdirectory, for example builds/linux/obs/alldeps/debian/changelog, using the syntax builds/linux/obs/alldeps/debian.changelog returns a No such file or directory error message.

However, using the syntax builds/linux/obs/alldeps/debian/changelog may not place changelog under the debian directory. Is this assumption correct ? I'm not able to evaluate this behaviour as all my Debian builds fail with unresolvable as described above.

Any thoughts ?

adrianschroeter · 2018-02-20T07:11:42Z

On Montag, 19. Februar 2018, 22:34:11 CET wrote Trevor SANDY:

Aside from the exceptions below, with this _service, all the images are **unresolvable** with either `nothing provides obs-service-tar`, `nothing provides obs-service-recompress`, or both returned messages...

buildtime services need to be available in the repository, yes. Only newer SUSE distros have these already, but you should find packages for all others in openSUSE:Tools project. So instead of building eg against Fedora:20/standard, build against openSUSE:Tools/Fedora_20

…

-- Adrian Schroeter email: adrian@suse.de SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany

trevorsandy · 2018-02-20T07:58:42Z

Ok - I'll give this a shot.

What about resolving the debian folder - does obs_scm automatically put all extracts at the root of the source directory as there does not appear to be any name="outfilename" option to direct output ?

With a location such as builds/linux/obs/alldeps/debian/changelog, using the syntax builds/linux/obs/alldeps/debian.changelog returns a No such file or directory error message.

However, using the syntax builds/linux/obs/alldeps/debian/changelog works but it places the changelog at the root of the source directory versus in the debian folder.

aspiers · 2018-06-19T10:37:40Z

Sorry I'm late to the party here.

@trevorsandy commented on 19 Feb 2018, 19:27 GMT:

I'll add the example you provided here so anyone looking for the same solution will have a better chance of finding it than I did.

That's a noble sentiment but unfortunately burying the info in a discussion in a closed issue doesn't do much to improve its chances of being discovered; it needs to be included in documentation. Unfortunately we don't have any docs yet except the README and the .service files. I've submitted #238 for that.

tacerus · 2022-08-21T20:27:51Z

Interesting to stumble across this issue. I was so far using the following in my RPM .spec file:

%define commit %(awk '/^commit/{print $2}' %{_sourcedir}/%{name}.obsinfo)

yvs2014 · 2024-08-04T21:37:47Z

You don't need extract_file usually, since you can tell obs_scm already to extract via the "extract" parameter.

there does not appear to be any name="outfilename" option to direct output ?

Hi everyone,
come across to the same need of renaming file (debian/file => debian.file) at extraction, is there a way to do that with "extract" parameter"?

trevorsandy closed this as completed Feb 19, 2018

trevorsandy reopened this Feb 19, 2018

aspiers mentioned this issue Jun 19, 2018

Needs proper user documentation #238

Open

4 tasks

aspiers added the documentation label Jun 19, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using obs_scm and extracting files in the same source _service #223

Using obs_scm and extracting files in the same source _service #223

trevorsandy commented Feb 19, 2018 •

edited

Loading

adrianschroeter commented Feb 19, 2018 via email

trevorsandy commented Feb 19, 2018 •

edited

Loading

trevorsandy commented Feb 19, 2018 •

edited

Loading

trevorsandy commented Feb 19, 2018

adrianschroeter commented Feb 20, 2018 via email

trevorsandy commented Feb 20, 2018

aspiers commented Jun 19, 2018

tacerus commented Aug 21, 2022

yvs2014 commented Aug 4, 2024

Using obs_scm and extracting files in the same source _service #223

Using obs_scm and extracting files in the same source _service #223

Comments

trevorsandy commented Feb 19, 2018 • edited Loading

Problem

Objective

Current configuraiton

Desired behaviour

Problem

Expected behaviour

adrianschroeter commented Feb 19, 2018 via email

trevorsandy commented Feb 19, 2018 • edited Loading

trevorsandy commented Feb 19, 2018 • edited Loading

trevorsandy commented Feb 19, 2018

adrianschroeter commented Feb 20, 2018 via email

trevorsandy commented Feb 20, 2018

aspiers commented Jun 19, 2018

tacerus commented Aug 21, 2022

yvs2014 commented Aug 4, 2024

trevorsandy commented Feb 19, 2018 •

edited

Loading

trevorsandy commented Feb 19, 2018 •

edited

Loading

trevorsandy commented Feb 19, 2018 •

edited

Loading