alpha version broken because of path changes in upstream repo

[endreigesund]

 | Cannot find path '/agent/_work/1/s/runner-images/images/linux' because
 | it does not exist.

[YannickRe]

Yes, this seems like a change that happened 4h ago. I will need to take the changes from https://github.com/actions/runner-images/blob/main/images.CI/linux-and-win/azure-pipelines/image-generation.yml and incorporate them into https://github.com/YannickRe/azuredevops-buildagents/blob/main/buildagent-generation-template.yml

[endreigesund]

Seems a lot going on at the moment - actions/runner-images#8706

Nice - do you have any eta of when you have time looking into it? We are waiting to roll out new images with .net 8 :) - current this is on prerelease and rolling out as we speak.

[erik-de-bont]

@endreigesund @YannickRe I have updated the buildagent-generation-template.yml on my fork. I am now building a new image an alpha release of ubuntu 2004, which seems to work. I need to do some more testing before I do a pull request, please have a look at the new version on my fork.

PS: It only works with the alpha versions, not with the prerelease or release versions, which makes sense since these versions use the old directory structure.

The new directory structure is now used by all images (release, pre-release and alpha)

[erik-de-bont]

Making progress. The Image building is working with the new directory structure setup. The generating a Gallery VM not yet. I think I have found the issue. Will run more tests,

[erik-de-bont]

Fixed the last issue. Was able to generate an new alpha ubuntu 22.04 image. I have made a pull request for the new code.

[endreigesund]

@erik-de-bont failed running from your fork now:

[endreigesund]

@erik-de-bont failed running from your fork now:

found that this was caused from my pipeline setting parameter "repository_base_path". Changed this to defulat "." and seems to be running now..

[erik-de-bont]

@endreigesund Thanks for the update. Let me know if works. My current status:

• Only works for alpha versions (version created 15 november and later) since the image make use of the new dir structure.
• Ubuntu image 22.04 created. Ubuntu image 20.04 currently running.
• Not been able to created to create Windows 2019 or 2022 yet. This doesn't seem to be related to newbuildagent-generation-template.yml, but due to some installation scripts during the build. A SHA doesn't match the downloaded installation file, causing the image build to fail.

[endreigesund]

since i'm running from devops i think must specify repository_base_path to azuredevops-buildagents or the repo won't downloading in this step:

When specified it fails in build vm step

When not specifying parameter "repository_base_path" it runs build vm, but then fails in create image gallery step:

So not able to use the new version from azure devops when templating pipeline from your repo

[erik-de-bont]

@endreigesund You should make your own fork or clone and run it from that repository to avoid this issue. I prefer a fork so I can sync updates from the main repo.

[endreigesund]

Understand that is a workable option, this has been working fine until now :)
https://github.com/erik-de-bont/azuredevops-buildagents#how-to-use

[erik-de-bont]

Strange. You should not have to configure 'repository_base_path' in your pipeline, since it's system based variable. The buildagent-generation.yml is the yaml you should use for your pipeline. I have configured my pipeline like this:

I understand the confusion now. I haven't read the README.MD documentation in while. It is not required to use the buildagent-generation-template.yml. You can use the buildagent-generation.yml straight away.

[ianjirka]

FWIW I'm having the same issue that @endreigesund is having when testing the changes from your branch.

I do think repostiory_base_path is needed here because I can't reference buildagent-generation.yml from my own YAML pipeline, because it is not a template, and so fails when the parser sees elements like 'trigger'.

[ianjirka]

On my build machine, the tree layout looks like this:

.
├── azuredevops-buildagents
│   ├── CODE_OF_CONDUCT.md
│   ├── README.md
...
├── runner-images

Eg, runner-images is placed as a peer directory, not inside the resource repo.

In a forked repo, I updated the call to build-image.ps1 to reference it as filePath: runner-images/images.CI/linux-and-win/build-image.ps1 - this allows buildVM to run. I'll know in a bit whether the rest of the pipeline runs as well.

I don't know if this will break the case where you invoke buildagent-generation.yaml directly, though. I don't have that setup.

[pippolino]

With the @erik-de-bont's PR code it works correctly on release ubuntu 22.04. I wanted to make sure there were no regressions on windows but I'm having problems with this an know issue 8884

[ianjirka]

In a forked repo, I updated the call to build-image.ps1 to reference it as filePath: runner-images/images.CI/linux-and-win/build-image.ps1 - this allows buildVM to run. I'll know in a bit whether the rest of the pipeline runs as well.

It took a while for me to validate because runner-images was broken for ubuntu for a bit. These changes build successfully for me now. I reference the -template.yaml from my own build pipeline because I created it back when that was the recommendation.

You can see the changes here. Unfortunately the diff is a bit ugly because I didn't catch that my auto-formatter ran :|.

@erik-de-bont, let me know what you think. I can make a formatterless PR change if it's useful to you, but can't validate the case where the pipeline is set up using build-generation.yaml directly.

[erik-de-bont]

@ianjirka I have started a build-image of ubuntu 2204 (release) and it building now, so your code seems also to work ok with using build-generation.yaml directly. The filepath in the 'buildagent-generation-template.yml' has now a relative path setting without the parameter.

@endreigesund @YannickRe @ianjirka
The code is also updated in the pull request, so I think the path issue should be sorted now.

PS: The 'release' images are now also using the new directory structure.

[endreigesund]

@erik-de-bont @ianjirka confirmed also from my side.. new paths solved this.

Do you normally run from alpha, prerelease or release tags? Seems to be a high failurerate on building this images.

[ianjirka]

@erik-de-bont @ianjirka confirmed also from my side.. new paths solved this.

Do you normally run from alpha, prerelease or release tags? Seems to be a high failurerate on building this images.

I only build release images. I noticed the image build pipeline failing when investigating why .Net 8 wasn't on the build machine.

[mvpwasilewskim]

Consider using alpha (main branch): actions/runner-images#8476 (comment)

[erik-de-bont]

@endreigesund Thanks for the update. Let me know if works. My current status:

• Only works for alpha versions (version created 15 november and later) since the image make use of the new dir structure.
• Ubuntu image 22.04 created. Ubuntu image 20.04 currently running.
• Not been able to created to create Windows 2019 or 2022 yet. This doesn't seem to be related to newbuildagent-generation-template.yml, but due to some installation scripts during the build. A SHA doesn't match the downloaded installation file, causing the image build to fail.

Now able to create all Linux and Windows images with new directory structure

[aolmosj]

Well, now there are even more changes, so it's broken again...

Attempting to modify C:\a\1\s\runner-images\images\windows\templates\windows-2019.json with workaround
Get-Content : Cannot find path 'C:\a\1\s\runner-images\images\windows\templates\windows-2019.json' because it does not 
exist.

They changed these json files, now they are hcl:

https://github.com/actions/runner-images/blob/main/images/windows/templates/windows-2022.pkr.hcl

[erik-de-bont]

@aolmosj Thanks for the heads up! I have updated the code in my fork (and Pull Request) to use pkr.hcl templates. I am able to build the alpha images again. Pre-release and release are still using the JSON format. It usually takes 2-4 weeks before the alpha images become a release, so we are again in a transition period.

[YannickRe]

@erik-de-bont 's PR has been merged, and that should fix this issue.