Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creates PIG Gradle plugin #129

Merged
merged 1 commit into from
Nov 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 27 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,33 @@ Both are available in [Maven Central](https://search.maven.org/search?q=partiql-

### Gradle

There are [plans to make a Gradle plugin for PIG](https://github.com/partiql/partiql-ir-generator/issues/102) but one
has not been completed yet.

Without the aforementioned plugin, the best way to use pig with gradle is:

- Add a dependency on PIG in your project's `buildSrc/build.gradle` file. This will make the API of PIG available to all
other `build.gradle` files in your project.
([Example](https://github.com/partiql/partiql-lang-kotlin/blob/main/buildSrc/build.gradle#L9))
- Add a dependency on PIG's runtime library in your project.
([Example](https://github.com/partiql/partiql-lang-kotlin/blob/28701e23cf3bd397a67e8d9ab4f68feff953aea1/lang/build.gradle#L48))
- Add a custom task that uses PIG's internal
APIs. ([Example](https://github.com/partiql/partiql-lang-kotlin/blob/51e7da7b5e63e45f01c4df101168b2117a17a2d1/lang/build.gradle#L64-L96))
- Make sure your custom task executes *before* the `compileKotlin` task.
([Example](https://github.com/partiql/partiql-lang-kotlin/blob/28701e23cf3bd397a67e8d9ab4f68feff953aea1/lang/build.gradle#L89))
```groovy
plugins {
id 'pig-gradle-plugin'
}

sourceSets {
main {
pig {
// in addition to the default 'src/main/pig'
srcDir 'path/to/type/universes'
}
}
test {
pig {
// in addition to the default 'src/test/pig'
srcDir 'path/to/test/type/universes'
}
}
}

pig {
target = 'kotlin' // required
namespace = ... // optional
template = ... // optional
outDir = 'build/generated-sources/pig/' // default
}
```

### Other Build Systems

Expand Down
55 changes: 55 additions & 0 deletions pig-gradle-plugin/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
plugins {
id("java-gradle-plugin")
id("org.jetbrains.kotlin.jvm") version "1.4.0"
id("com.gradle.plugin-publish") version "1.0.0"
}

repositories {
mavenCentral()
}

version = "0.5.1-SNAPSHOT"
group = "org.partiql"

dependencies {
// It is non-trivial to depend on a local plugin within a gradle project
// The simplest way is using a composite build: https://docs.gradle.org/current/userguide/composite_builds.html
// Other methods involved adding the build/lib/... jar to classpath, or publish to maven local
// By adding the plugin as a dep in `pig-tests`, I cannot use an included build of `pig` in the plugin
// Hence it's much simpler to use the latest published version in the plugin
implementation("org.partiql:partiql-ir-generator:0.5.0")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed in-person, but just leaving the note here as well for others to discuss. Might be worth looking into alternatives to hard-coding the version. We discussed referencing the JAR if possible. I tried pulling this and going another route but had some issues. Anyways, glad you found a way to make it work, but would be nice to remove the hard-code!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plugin, runtime, and generator should all be a single gradle project. The issue here is caused by the pig-tests package which, as is, should not be in the main pig gradle project. The pig-tests needs to be fixed to use the generator directly rather than the executable. Executable/plugin usage should be in a separate "pig-example" project which would avoid these oddities. Right now, example usage and testing are intermixed. #130

testImplementation("org.junit.jupiter:junit-jupiter-api:5.8.1")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.8.1")
}

tasks.test {
useJUnitPlatform()
}

pluginBundle {
website = "https://github.com/partiql/partiql-ir-generator/wiki"
vcsUrl = "https://github.com/partiql/partiql-ir-generator"
tags = listOf("partiql", "pig", "ir", "partiql-ir-generator")
}

gradlePlugin {
plugins {
create("pig-gradle-plugin") {
id = "pig-gradle-plugin"
displayName = "PIG Gradle Plugin"
description = "The PIG gradle plugin exposes a Gradle task to generate sources from a PIG type universe"
implementationClass = "org.partiql.pig.plugin.PigPlugin"
}
}
}

//
// // TODO https://github.com/partiql/partiql-ir-generator/issues/132
// publishing {
// repositories {
// maven {
// name = 'mavenLocalPlugin'
// url = '../maven-local-plugin'
// }
// }
// }
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
package org.partiql.pig.plugin

import org.gradle.api.Project
import org.gradle.api.provider.Property
import javax.inject.Inject

abstract class PigExtension @Inject constructor(project: Project) {

private val objects = project.objects

val conventionalOutDir: String

init {
conventionalOutDir = "${project.buildDir}/generated-sources/pig"
}

// required
val target: Property<String> = objects.property(String::class.java)

// optional
val outputFile: Property<String> = objects.property(String::class.java)

// optional
val outputDir: Property<String> = objects
.property(String::class.java)
.convention(conventionalOutDir)

// optional
val namespace: Property<String> = objects.property(String::class.java)

// optional
val template: Property<String> = objects.property(String::class.java)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
package org.partiql.pig.plugin

import org.gradle.api.Plugin
import org.gradle.api.Project
import org.gradle.api.file.SourceDirectorySet
import org.gradle.api.plugins.JavaPlugin
import org.gradle.api.tasks.SourceSet
import org.gradle.api.tasks.SourceSetContainer

abstract class PigPlugin : Plugin<Project> {

override fun apply(project: Project) {
// Ensure `sourceSets` extension exists
project.pluginManager.apply(JavaPlugin::class.java)

// Adds pig source set extension to all source sets
project.sourceSets().forEach { sourceSet ->
val name = sourceSet.name
val sds = project.objects.sourceDirectorySet(name, "$name PIG source")
sds.srcDir("src/$name/pig")
sds.include("**/*.ion")
sourceSet.extensions.add("pig", sds)
}

// Extensions for pig compiler arguments
val ext = project.extensions.create("pig", PigExtension::class.java, project)

// Create tasks after source sets have been evaluated
project.afterEvaluate {
project.sourceSets().forEach { sourceSet ->
// Pig generate all for the given source set
val pigAllTaskName = getPigAllTaskName(sourceSet)
val pigAllTask = project.tasks.create(pigAllTaskName) {
it.group = "pig"
it.description = "Generate all PIG sources for ${sourceSet.name} source set"
}

// If outDir is conventional, add generated sources to javac sources
// Else you're responsible for your own configuration choices
var outDir = ext.outputDir.get()
if (outDir == ext.conventionalOutDir) {
outDir = outDir + "/" + sourceSet.name
sourceSet.java.srcDir(outDir)
}

// Create a pig task for each type universe and each source set
(sourceSet.extensions.getByName("pig") as SourceDirectorySet).files.forEach { file ->
val universeName = file.name.removeSuffix(".ion").lowerToCamelCase().capitalize()
val pigTask = project.tasks.create(pigAllTaskName + universeName, PigTask::class.java) { task ->
task.description = "Generated PIG sources for $universeName"
task.universe.set(file.absolutePath)
task.target.set(ext.target)
task.outputDir.set(outDir)
task.outputFile.set(ext.outputFile)
task.namespace.set(ext.namespace)
task.template.set(ext.template)
}
pigAllTask.dependsOn(pigTask)
}

// Execute pig tasks before compiling
project.tasks.named(sourceSet.compileJavaTaskName) {
it.dependsOn(pigAllTask)
}
}
}
}

private fun Project.sourceSets(): List<SourceSet> = extensions.getByType(SourceSetContainer::class.java).toList()

private fun getPigAllTaskName(sourceSet: SourceSet) = when (sourceSet.name) {
"main" -> "generatePigSource"
else -> "generatePig${sourceSet.name.capitalize()}Source"
}

/**
* Type Universe files are lower hyphen, but Gradle tasks are lower camel
*/
private fun String.lowerToCamelCase(): String =
this.split('-')
.filter { it.isNotEmpty() }
.mapIndexed { i, str ->
when (i) {
0 -> str
else -> str.capitalize()
}
}
.joinToString(separator = "")
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package org.partiql.pig.plugin

import org.gradle.api.DefaultTask
import org.gradle.api.provider.Property
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.Optional
import org.gradle.api.tasks.TaskAction
import org.gradle.api.tasks.options.Option

abstract class PigTask : DefaultTask() {

init {
group = "pig"
}

@get:Input
@get:Option(
option = "universe",
description = "Type universe input file"
)
abstract val universe: Property<String>

@get:Input
@get:Option(
option = "target",
description = "Target language"
)
abstract val target: Property<String>

@get:Input
@get:Optional
@get:Option(
option = "outputFile",
description = "Generated output file (for targets that output a single file)"
)
abstract val outputFile: Property<String>

@get:Input
@get:Optional
@get:Option(
option = "outputDir",
description = "Generated output directory (for targets that output multiple files)"
)
abstract val outputDir: Property<String>

@get:Input
@get:Optional
@get:Option(
option = "namespace",
description = "Namespace for generated code"
)
abstract val namespace: Property<String>

@get:Input
@get:Optional
@get:Option(
option = "template",
description = "Path to an Apache FreeMarker template"
)
abstract val template: Property<String>

@TaskAction
fun action() {
val args = mutableListOf<String>()
// required args
args += listOf("-u", universe.get())
args += listOf("-t", target.get())
// optional args
if (outputFile.isPresent) {
args += listOf("-o", outputFile.get())
}
if (outputDir.isPresent) {
args += listOf("-d", outputDir.get())
}
if (namespace.isPresent) {
args += listOf("-n", namespace.get())
}
if (template.isPresent) {
args += listOf("-e", template.get())
}
// invoke pig compiler, offloads all arg handling to the application
// also invoking via the public interface for consistency
println("pig ${args.joinToString(" ")}")
org.partiql.pig.main(args.toTypedArray())
}
}
Loading