Random "Invalid argument(s): Missing library [...]" when run build_runner #3657
Comments
bummsa
changed the title
|
flutter doctor -v [✗] Android toolchain - develop for Android devices [✓] Xcode - develop for iOS and macOS (Xcode 15.2) [✓] Chrome - develop for the web [✓] Android Studio (version 2023.1) [✓] IntelliJ IDEA Community Edition (version 2023.3.3) [✓] Connected device (2 available) [✓] Network resources |
|
Running build_runner simultaneously in multiple projects that depend on each other is not supported, they don't know about each other and will end up deleting files out from under other instances in the middle of their builds. |
Hi, but in my case the build_runner do not run simultaneously, they run one after the other |
|
@jakemac53 can you please review the case again? Thank you in advance! |
|
Ah, you are manually doing a |
|
Also if you can run with |
Yes - The build runner is executed by Jenkins. Basically, I just check all folders within "my_app" and whether the file "pubspec.yaml" exists in the subfolder (shared_project, the app project). And then it checks if the pattern "build_runner" was found in the pubspec.yaml. Could it be a problem with the order in which we call the build_runner? Here's a small piece of code from the jenkins pipeline:
|
[FINE] json_serializable on lib/src/workorder_search/ui/workorder_search_list_tile_full.dart:Running JsonSerializableGenerator - 1 of 2 Invalid argument(s): Missing library: package:shared_project_1/enum/filter_enums.dart here it is. |
|
The order could matter yes - especially if you are not checking generated files into git. Maybe it is not deterministic which order packages are seen? |
Good morning, My project is pretty big, but here is an advanced example: My app and shared_3 refer to shared_1 and shared_2, which means I have to build 1 and 2 first. Do you have any other idea? |
|
@jakemac53 Can you please review it again? Many thanks in advance! |
|
I don't have any particular idea here, other than the symptom is what you would see when parallel builds happen. Are you sure Jenkins isn't running those steps in parallel? I don't know anything about Jenkins. |
|
I've also been seeing intermittent failure with the same "Invalid argument(s): Missing library" error. |
|
Verbose build for review
[INFO] Entrypoint:Generating build script...
[INFO] Entrypoint:Generating build script completed, took 172ms
[INFO] BuildDefinition:Initializing inputs [INFO] BuildDefinition:Checking for updates since last build... [INFO] Build:Running build... [INFO] Build:Caching finalized dependency graph... [SEVERE] dart_json_mapper on lib/main_preview.dart (cached): Invalid argument(s): Missing library: package:zory/providers.dart package:analyzer/src/summary2/linked_element_factory.dart 153:7 LinkedElementFactory.createLibraryElementForReading [SEVERE] Build: |
May I ask which OS you use? |
|
Here's a list what I've already tried:
The best way might be to commit the generated files? I develop the app under Windows and have no problems here. The problem only occurs when developing/building under MacOS. |
|
Mac OS Sonoma 14.4 |
|
Can we please open this back up? I think the issue is wider than originally reported. |
|
@matt-hall-zory in your scenario is there just a single package that you are building? |
|
Yeah, I'm just building the Zory App, but I'm regularly seeing the same "Missing Library" error as listed above. |
|
Ok, I will re-open given it can manifest in a single package scenario. Not sure if @scheglov has any ideas here. Possibly there has been some change where some behavior is now asynchronous which wasn't before, when adding new files to an analysis context? |
|
I don't remember any recent changes here. I see that To avoid this issue you need to do it sequentially:
|
|
Forgive my ignorance but I'm completely baffled by your reply. I'm using a number of packages (riverpod, dart_json_mapper and auto_route) which generate code. Whenever I make changes (add/remove/modify models, files, routes) I run dart run build_runner build --delete-conflicting-outputs. |
|
I think his response was for the build_runner authors than for users :). |
Unfortunately due to the nature of how build_runner works, I just don't think this at all possible... we have any number of "parallel" build steps going at once, this is a pull-based process and so we can't just delay sending the file updates to the analyzer until some later time, even though we do have "phases" which would be natural split points, we don't run those phases in their entirety before running the next phase, they are only used to establish an ordering between individual build steps. Essentially, lets consider a 2 layer build, where builders A/B run in "phases" A/B respectively (where phase B is after phase A), on files 1.dart and 2.dart. Lets also say that each of those phases is generating a part file which is included in the library the builder is applied to, and that file 1.dart imports 2.dart. We start by building the final output, so from the latest phase (phase B). We start running that builder (B) on all the files in that phase (1.dart and 2.dart) at the same time. These both request a LibraryElement, which will wait for any transitive inputs from previous phases to be built before it can return. So, on 2.dart that waits for the part file (2.a.dart), which we immediately start building. And for 1.dart it waits for both 1.a.dart and 2.a.dart since it depends on both. So phase A is now also building, running also on 1.dart and 2.dart. Those each resolve their libraries as well, but don't depend on generated files so they are given a library element (note that some of the included parts, 1.b.dart and 2.b.dart are not yet present though). If 2.a.dart finishes first, we call I see why this situation is problematic, but it used to work out fine, as long as you didn't reach out and try to use the analysis driver directly, old LibraryElement(s) were still valid? |
Not quite. We load libraries from summaries incrementally, so that we don't spend time and heap loading elements that are never used. But we make sure that when we need something, it is available in form of Here is the code that demonstrates the scenario. solo_test_changeFile_XXX() async {
final a = newFile('$testPackageLibPath/a.dart', r'''
class A {}
''');
final b = newFile('$testPackageLibPath/b.dart', r'''
import 'a.dart';
A foo() {}
''');
// Build summaries.
{
final driver = driverFor(b);
await driver.getLibraryByUri('package:test/b.dart');
}
await disposeAnalysisContextCollection();
// Get the driver that reads libraries from summaries.
final driver = driverFor(b);
// This will make sure that `a` and `b` bundles are loaded for reading.
// And then read `b` bundle into the element.
final bResult = await driver.getLibraryByUri('package:test/b.dart');
bResult as LibraryElementResult;
final bUnit = bResult.element.definingCompilationUnit;
final foo = bUnit.functions.single;
// Notify that `a` changed.
driver.changeFile2(a);
// Give the driver enough time to process all scheduled events.
// This will unload `a` and `b`.
// Now we hold an invalidated `b` element.
await pumpEventQueue(times: 5000);
// This will need to instantiate `A` class.
// It will fail because `a` is not loaded anymore.
print(foo.returnType);
} |
|
Yeah, I do understand the issue at play here. It seems to be new though? Or somehow only manifests now when it didn't used to. I haven't ever seen this error reported until this issue. And, I don't have any proposal for a fix. It seems fundamentally at odds with how build_runner does builds. |
|
It's probably not useful for you all but probably useful for any developers who stumble on this in the meantime. I found a workaround. |
I've got a similar issue, and @matt-hall-zory tip is working fine as well on my side. @jakemac53 you said we can't run simultaneously build_runner, so I'm wondering about a single run on these kind of files I've got on a first file : And on a second file : Please note that everything was working fine before... (when it starts to fail? too difficult to say ....) |
Hi there,
I'm having an issue building my Flutter app for iOS for about every 2-3 builds during code generation and it occurs randomly and I can't recognize a pattern.
My project structure looks like this:
my_app
--> shared_project_1
--> shared_project_2
--> application_project
I run the build_runner in all projects and it fails randomly in any of the 3 subprojects and any dto's I have in the projects with the following error:
Invalid argument(s): Missing library: package:shared_project_1/shared/dto/fruit_dto.dart Libraries: [dart:async, dart:collection, dart:convert, dart:core, ...]and the fruit_dto looks like this:
`import ...;
part 'fruit_dto.g.dart';
@JsonSerializable()
class FruitDto extends Dto {
final String name;
final String shape;
final String color;
FruitDto({
int? id,
int? noi,
super.entityTag,
required this.name,
required this.shape,
this.color = '',
}) : super(containerId: id, noI: noi);
static FruitDto fromJson(Map<String, dynamic> json) =>
_$FruitDtoFromJson(json);
@OverRide
Map<String, dynamic> toJson() => _$FruitDtoToJson(this);
}`
It lists like all the classes I have in my project, including some Flutter packages, such as "package:flutter/src/foundation/print.dart".
flutter sdk: 3.19.3 (reproducible with 3.16.x as well)
build_runner: 2.4.8
json_serializable: 6.7.1
Mac Mini with M2 Pro: 14.2.1
Does anyone have any ideas on how to fix the issue?
Thanks and best regards
bummsa
The text was updated successfully, but these errors were encountered: