Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging of journalbeat #8702

Merged
merged 17 commits into from
Oct 24, 2018
6 changes: 2 additions & 4 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,12 +152,10 @@ addons:
apt:
update: true
packages:
- libc6-dev-i386
- libpcap-dev
- libsystemd-journal-dev
- libxml2-utils
- python-virtualenv
- libpcap-dev
- xsltproc
- libxml2-utils

before_install:
- python --version
Expand Down
9 changes: 7 additions & 2 deletions dev-tools/mage/crossbuild.go
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ func buildMage() error {
return sh.RunWith(env, "mage", "-f", "-compile", filepath.Join("build", "mage-linux-amd64"))
}

func crossBuildImage(platform string) (string, error) {
func crossBuildImage(platform, beatName string) (string, error) {
tagSuffix := "main"

switch {
Expand All @@ -167,6 +167,10 @@ func crossBuildImage(platform string) (string, error) {
// Use an older version of libc to gain greater OS compatibility.
// Debian 7 uses glibc 2.13.
tagSuffix = "main-debian7"
// journalbeat requires Debian 8 due to systemd support.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly all logic in the mage package is agnostic of the beat names. I think you keep this logic in the journalbeat magefile if you add a new CrossBuildOption that allows you to pass in your own "ImageSelector" func. If none is set then it should default to using mage.crossBuildImage.

type ImageSelectorFunc func(platform string) (string, error)

func ImageSelector(f ImageSelectorFunc) func(params *crossBuildParams) {
	return func(params *crossBuildParams) {
		params.ImageSelector = f
	}
}

if beatName == "journalbeat" {
tagSuffix = "main-debian8"
}
}

goVersion, err := GoVersion()
Expand Down Expand Up @@ -208,7 +212,8 @@ func (b GolangCrossBuilder) Build() error {
}

dockerRun := sh.RunCmd("docker", "run")
image, err := crossBuildImage(b.Platform)
beatName := filepath.Base(repoInfo.RootDir)
image, err := crossBuildImage(b.Platform, beatName)
if err != nil {
return errors.Wrap(err, "failed to determine golang-crossbuild image tag")
}
Expand Down
2 changes: 0 additions & 2 deletions journalbeat/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ BEAT_TITLE=Journalbeat
SYSTEM_TESTS=false
TEST_ENVIRONMENT=false
ES_BEATS?=..
GOX_FLAGS=-cgo
GOX_OS=linux

# Path to the libbeat Makefile
-include $(ES_BEATS)/libbeat/scripts/Makefile
Expand Down
8 changes: 8 additions & 0 deletions journalbeat/journalbeat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,14 @@ output.elasticsearch:
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"

#================================ Procesors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
- add_host_metadata: ~
- add_cloud_metadata: ~

#================================ Logging =====================================

# Sets log level. The default log level is info.
Expand Down
11 changes: 10 additions & 1 deletion journalbeat/magefile.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import (
func init() {
mage.BeatDescription = "Journalbeat ships systemd journal entries to Elasticsearch or Logstash."

// TODO filter platforms
mage.Platforms = mage.Platforms.Filter("linux")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given then ppc64 and mips64 can't be build at , I'm wondering if we should filter them like linux !linux/ppc64 !linux/mips64. The downside is that it will silently ignore those platforms, but the upside is that you can build the all of beats with PLATFORMS='+all linux/mips' make snapshot and it will just work.

}

// Build builds the Beat binary.
Expand All @@ -44,9 +44,18 @@ func Build() error {
// GolangCrossBuild build the Beat binary inside of the golang-builder.
// Do not use directly, use crossBuild instead.
func GolangCrossBuild() error {
mg.Deps(installDependencies)
return mage.GolangCrossBuild(mage.DefaultGolangCrossBuildArgs())
}

func installDependencies() error {
if err := sh.Run("apt-get", "update"); err != nil {
return err
}

return sh.Run("apt-get", "install", "-y", "--no-install-recommends", "libsystemd-dev")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work for linux/386? I'd assume that you need to run specific commands for each architecture like apt-get install libsystemd-dev:i386.

Then for armv7 and all other architectures you'd need something like dpkg --add-architecture armhf && apt-get update && apt-get install libsystemd-dev:armhf.

}

// BuildGoDaemon builds the go-daemon binary (use crossBuildGoDaemon).
func BuildGoDaemon() error {
return mage.BuildGoDaemon()
Expand Down
2 changes: 2 additions & 0 deletions libbeat/scripts/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,11 @@ ${BEAT_NAME}.test: $(GOFILES_ALL)
.PHONY: crosscompile
crosscompile: ## @build Cross-compile beat for the OS'es specified in GOX_OS variable. The binaries are placed in the build/bin directory.
crosscompile: $(GOFILES)
ifneq ($(shell [[ $(BEAT_NAME) == journalbeat ]] && echo true ),true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? How about removing the thing that's invoking crosscompile for journalbeat (I assume this can just be dropped from the travis config).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crosscompile was not added to Travis, so there was nothing to remove. I added the conditional here, so if someone tries to run it locally, it is skipped.

go get github.com/mitchellh/gox
mkdir -p ${BUILD_DIR}/bin
gox -output="${BUILD_DIR}/bin/{{.Dir}}-{{.OS}}-{{.Arch}}" -os="$(strip $(GOX_OS))" -osarch="$(strip $(GOX_OSARCH))" ${GOX_FLAGS}
endif

.PHONY: check
check: check-headers python-env prepare-tests ## @build Checks project and source code if everything is according to standard
Expand Down
17 changes: 17 additions & 0 deletions x-pack/journalbeat/cmd/root.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License;
// you may not use this file except in compliance with the Elastic License.

package cmd

import (
"github.com/elastic/beats/journalbeat/cmd"
xpackcmd "github.com/elastic/beats/x-pack/libbeat/cmd"
)

// RootCmd to handle beats cli
var RootCmd = cmd.RootCmd

func init() {
xpackcmd.AddXPack(RootCmd, cmd.Name)
}
19 changes: 19 additions & 0 deletions x-pack/journalbeat/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License;
// you may not use this file except in compliance with the Elastic License.

package main

import (
"os"

"github.com/elastic/beats/x-pack/journalbeat/cmd"

_ "github.com/elastic/beats/journalbeat/include"
)

func main() {
if err := cmd.RootCmd.Execute(); err != nil {
os.Exit(1)
}
}