CompilerGym v0.2.4 #686

CompilerGym v0.2.3

…apper

Refactor out Env interface from CompilerEnv

… in ClientServiceCompilerEnv

…conversion

Make action spaces and action service message conversion configurable in ClientServiceCompilerEnv

`go get` is deprecated, replace with `go install`.

Include name in space comparison

Fix pre-commit CI job

Add dispatching on type_id during action and observation conversion

Add permutation and space sequence spaces

Co-authored-by: kyle <kyle.w.herndon@gmail.com>

Update rulefiles_path.py to align cache path with c file definition.

Add a "News & Events" section to README

…e_process

…spaces

The system compiler could fail to compile a simple file within the time limit (30 seconds). Add a retry loop to attempt this three times before giving up.

Update runfiles_path.py

Modified the env.reset() function to add the reward_space and observation_space parameters. Modified the reset() func of the subclasses. Added a OptionalArgumentValue class in gym_type_hints.py for the default value.

Added some simple unit test to test if the modified reset() function is working as intended.

…ntime dependence

CI hardening patchset

Add reward_space and observation_space parameters to env.reset()

…om Ray

Improve the exec logging message.

Fix Py < 3.8 compatibility for shlex.join()

This adds docstrings to cover the new reward_space and observation_space arguments to reset(). Fixes #451.

…pported action spaces

…envs.mlir

Bump absl and gRPC dependencies

Use the version of clang that is bundled with CompilerGym, since the user may not have a compatible version of clang in their $PATH.

[examples] Fix path to clang in example services.

Misc fixes and improvements

[examples] Install *.yaml files for llvm_rl package.

Add abstract declarations of the reset / constructor operators to the base CompilerEnv class so that the docstrings appear in the right place. This also adds an FAQ entry to further elaborate on the reason for needing to call env.close().

Likely a symptom of #648.

[docs] New FAQ questions "Where does CompilerGym store files?" and "Do I need to call env.close()?"

The CMake installation may be the first to create ~/.local/bin, so it would not be in PATH without logging in again. This change makes sure that it is added to PATH in the same shell session.

In install instructions source ~/.profile

Create an MLIR environment with matrix multiplication

apt-get does not complain about missing clang++-9 because it treats it as a regex and is able to match other packages with it. apt does not use regex and fails with a missing package.

This fixes dependency issues where a dependency of a custom command on a custom target guarantees only the order of building and does not guarantee that the custom command would rerun every time when the custom target is rebuilt. The approach to fix this is for custom targets to record in the OUTPUTS target property their output files. Then the dependent custom command depends on the contents of the OUTPUTS property using generator expressions. This creates a file level dependencies that triggers rebuilds on change.

Fix install insturctions and build/ci

…ncies CMake: fix custom target dependencies

Remove most of the irrelevant CUDA / PyTorch deps.

This replaces the connection.working_dir property with a new class that will allow more sophisticated behaviors.

This changes the per-benchmark cache to be contained within the working directory of the parent service, rather than in the global cache.

This adds a new API for constructing a benchmark from a command line invocation of a compiler. For example: >>> benchmark = env.make_benchmark_from_command_line( ... ["gcc", "in.c", "-DNDEBUG"] ... ) This provides control over the build, equivalent to: >>> benchmark = env.make_benchmark(["in.c"], copts=["-NDEBUG"]) The idea is to make it easier to integrate CompilerGym into an existing build system, as it is normally possible to dump a verbose log of all of the compiler invocations run during a build.

This enables commandlines with a combination of source and objectfile inputs to be supported.

Some small cmake tweaks

[llvm] Move the per-benchmark scratch directory into the service working directory

Add a wrapper to synchronously log environment state transitions to SQLite database

This adds a new `Counter` wrapper that counts the number of calls to operations. Example usage: >>> env = Counter(compiler_gym.make("llvm-v0")) >>> env.counters {"close": 0, "reset": 0, "step": 0, "fork": 0} >>> env.step(0) {"close": 0, "reset": 0, "step": 1, "fork": 0}

This adds a new wrapper class, `ForkOnStep`, that can be used to backtrack by maintaining a stack of forked environments before every call to step().

[wrappers] Add a ForkOnStep wrapper.

…e-template Update bug fix issue template

[wrappers] Add a Counter wrapper class.

[llvm] Add API to construct a benchmark from clang invocation.

[Minor Bug] fix typo in argument & add flag

Update citation to CGO.

github.com//issues/688

@sogartar

This release adds a new compiler environment, new APIs, and a suite of backend improvements to improve the flexibility of CompilerGym environments. Many thanks to code contributors: @sogartar, @KyleHerndon, @SoumyajitKarmakar, @uduse, and @anthony0727! Highlights of this release include: - [mlir] Began work on a new environment for matrix multiplication using MLIR ([#652](#652), thanks @KyleHerndon and @sogartar!). Note this environment is not yet included in the pypi package and must be [compiled from source](https://github.com/facebookresearch/CompilerGym/blob/development/INSTALL.md#building-from-source-with-cmake). - [llvm] Added a new `env.benchmark_from_clang_invocation()` method ([#577](#577)) that can be used for constructing LLVM environment automatically from C/C++ compiler invocations. This makes it much easier to integrate CompilerGym with your existing build scripts. - Added three new wrapper classes: `Counter`, that provides op counts for analysis ([#683](#683)); `SynchronousSqliteLogger`, that provides logging of environment interactions to a relational database ([#679](#679)), and `ForkOnStep` that provides an `undo()` operation ([#682](#682)). - Added `reward_space` and `observation_space` parameters to `env.reset()` ([#659](#659), thanks @SoumyajitKarmakar!) This release includes a number of improvements to the backend APIs that make it easier to write new CompilerGym environments: - Refactored the backend to make `CompilerEnv` an abstract interface, and `ClientServiceCompilerEnv` the concrete implementation of this interface. This enables new environments to be implemented without using gRPC ([#633](#633), thanks @sogartar!). - Extended the support for different types of action and observation spaces ([#641](#641), [#643](#643), thanks @sogartar!), including new `Permutation` and `SpaceSequence` spaces ([#645](#645), thanks @sogartar!).. - Added a new `disk/` subdirectory to compiler service's working directories, which is symlinked to an on-disk location for devices which support in-memory working directories. This fixes a bug with leftover temporary directories from LLVM ([#672](#672)). This release also includes numerous bug fixes and improvements, many of which were reported or fixed by the community. For example, fixing a bug in cache file locations ([#656](#656), thanks @uduse!), and a missing flag definition in example code ([#684](#684), thanks @anthony0727!). **Full Changelog**: v0.2.3...v0.2.4 This release brings in deprecating changes to the core `env.step()` routine, and lays the groundwork for enabling new types of compiler optimizations to be exposed through CompilerGym. Many thanks to code contributors: @mostafaelhoushi, @sogartar, @KyleHerndon, @uduse, @parthchadha, and @xtremey! Highlights of this release include: - Added a new `TextSizeInBytes` observation space for LLVM ([#575](#575)). * Added a new PPO leaderboard entry ([#580](#580). Thanks @xtremey! - Fixed a bug in which temporary directories created by the LLVM environment were not cleaned up ([#592](#592)). - **[Backend]** The function `createAndRunCompilerGymService` now returns an int, which is the exit return code ([#592](#592)). - Improvements to the examples documentation ([#548](#548)) and FAQ ([#586](#586)) Deprecations and breaking changes: - `CompilerEnv.step` no longer accepts a list of actions ([#627](#627)). A new method, `CompilerEnv.multistep` provides this functionality. This is to provide compatibility with environments whose action spaces are lists. To update your code, replace any calls to `env.step()` which take a list of actions to use `env.multistep()`. Thanks @sogartar! - The arguments `observations` and `rewards` to `step()` have been renamed `observation_spaces` and `reward_spaces`, respectively ([#627](#627)). - `Reward.id` has been renamed `Reward.name` ([#565](#565), [#612](#612)). Thanks @parthchadha! * The backend protocol buffer schema has been updated to natively support more types of observation and action, and to support nested spaces ([#531](#531)). Thanks @sogartar!

Commits on Apr 25, 2022

Add compiler_gym.errors to setup.py

sogartar committed Apr 25, 2022

Configuration menu

View commit details

Copy full SHA for 85f74de

Browse repository at this point

Copy the full SHA

85f74de View commit details

Browse the repository at this point in the history

CompilerGym v0.2.4 #686

CompilerGym v0.2.4 #686

Commits on Mar 18, 2022

Commits on Mar 22, 2022

Commits on Mar 23, 2022

Commits on Mar 24, 2022

Commits on Mar 25, 2022

Commits on Mar 26, 2022

Commits on Mar 28, 2022

Commits on Mar 29, 2022

Commits on Mar 30, 2022

Commits on Mar 31, 2022

Commits on Apr 2, 2022

Commits on Apr 4, 2022

Commits on Apr 6, 2022

Commits on Apr 8, 2022

Commits on Apr 13, 2022

Commits on Apr 14, 2022

Commits on Apr 18, 2022

Commits on Apr 19, 2022

Commits on Apr 20, 2022

Commits on Apr 21, 2022

Commits on Apr 22, 2022

Commits on Apr 23, 2022

Commits on Apr 25, 2022

Commits on Apr 26, 2022

Commits on Apr 27, 2022

Commits on Apr 28, 2022

Commits on Apr 29, 2022

Commits on May 2, 2022

Commits on May 3, 2022

Commits on May 4, 2022

Commits on May 5, 2022

Commits on May 6, 2022

Commits on May 8, 2022

Commits on May 9, 2022

Commits on May 10, 2022

Commits on May 12, 2022

Commits on May 15, 2022

Commits on May 16, 2022

Commits on May 17, 2022

Commits on May 19, 2022

Commits on May 20, 2022

Commits on May 24, 2022