Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vanilla vigilante codebase #1

Merged
merged 28 commits into from
Aug 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
d6a6dff
btc client
SebastianElvis Aug 4, 2022
4d9c96e
rpc server
SebastianElvis Aug 4, 2022
0ee86e3
vigilantes
SebastianElvis Aug 4, 2022
19c0a6c
hello world vigilantes
SebastianElvis Aug 4, 2022
f9f8ac6
some refactor
SebastianElvis Aug 4, 2022
f801410
some init stuff
SebastianElvis Aug 4, 2022
4d83f34
log and main programs
SebastianElvis Aug 5, 2022
a3212a5
rpc server working
SebastianElvis Aug 5, 2022
d112653
cleanup config
SebastianElvis Aug 5, 2022
bf3b493
config for vigilantes
SebastianElvis Aug 5, 2022
8d21f37
tls for btc client
SebastianElvis Aug 5, 2022
91f849e
cleanup
SebastianElvis Aug 5, 2022
263a6cd
typo
SebastianElvis Aug 5, 2022
cb78621
read config file
SebastianElvis Aug 8, 2022
1c373f2
refactor
SebastianElvis Aug 8, 2022
fb17b8b
CLI flag for config file, and sample config
SebastianElvis Aug 9, 2022
5fc85d6
log for submodules
SebastianElvis Aug 9, 2022
df9e71f
typo fix
SebastianElvis Aug 9, 2022
cafe6dd
use btcwallet as dependency
SebastianElvis Aug 10, 2022
df55428
copyright
SebastianElvis Aug 10, 2022
4fc7bb7
censor directory
SebastianElvis Aug 10, 2022
22f5da0
version
SebastianElvis Aug 10, 2022
779c6e1
prometheus metrics
SebastianElvis Aug 10, 2022
53e49d0
lint log outputs
SebastianElvis Aug 10, 2022
3c64ae6
multiple cleanups
SebastianElvis Aug 10, 2022
e1e7cc8
server's access to submitter/reporter
SebastianElvis Aug 10, 2022
ead15ca
graceful shutdown of vigilante
SebastianElvis Aug 10, 2022
bc83eb4
prometheus middleware for grpc
SebastianElvis Aug 11, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,6 @@

# Dependency directories (remove the comment below to include it)
# vendor/

.vscode/
main
17 changes: 17 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
ISC License

Copyright (c) 2022-2022 The Babylon developers
Copyright (c) 2013-2017 The btcsuite developers
Copyright (c) 2015-2016 The Decred developers
Comment on lines +3 to +5
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤟


Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
3 changes: 3 additions & 0 deletions btcclient/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# btcclient

This package implements a Bitcoin client. The code is adapted from https://github.com/btcsuite/btcwallet/tree/master/chain.
58 changes: 58 additions & 0 deletions btcclient/client.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// Copyright (c) 2022-2022 The Babylon developers
// Copyright (c) 2013-2016 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.

package btcclient

import (
"errors"

"github.com/babylonchain/vigilante/config"
"github.com/babylonchain/vigilante/netparams"
"github.com/btcsuite/btcd/chaincfg"
"github.com/btcsuite/btcwallet/chain"
)

var _ chain.Interface = &Client{}

// Client represents a persistent client connection to a bitcoin RPC server
// for information regarding the current best block chain.
type Client struct {
*chain.RPCClient
Params *chaincfg.Params
Cfg *config.BTCConfig
}

// New creates a client connection to the server described by the
// connect string. If disableTLS is false, the remote RPC certificate must be
// provided in the certs slice. The connection is not established immediately,
// but must be done using the Start method. If the remote server does not
// operate on the same bitcoin network as described by the passed chain
// parameters, the connection will be disconnected.
func New(cfg *config.BTCConfig) (*Client, error) {
if cfg.ReconnectAttempts < 0 {
return nil, errors.New("reconnectAttempts must be positive")
}

certs := readCAFile(cfg)
params := netparams.GetBTCParams(cfg.NetParams)

rpcClient, err := chain.NewRPCClient(params, cfg.Endpoint, cfg.Username, cfg.Password, certs, cfg.DisableClientTLS, cfg.ReconnectAttempts)
if err != nil {
return nil, err
}
client := &Client{rpcClient, params, cfg}
return client, err
}

func (c *Client) ConnectLoop() {
go func() {
log.Infof("Start connecting to the BTC node %v", c.Cfg.Endpoint)
if err := c.Start(); err != nil {
log.Errorf("Unable to connect to the BTC node: %v", err)
}
log.Info("Successfully connected to the BTC node")
c.WaitForShutdown()
}()
}
7 changes: 7 additions & 0 deletions btcclient/log.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package btcclient

import (
vlog "github.com/babylonchain/vigilante/log"
)

var log = vlog.Logger.WithField("module", "btcclient")
26 changes: 26 additions & 0 deletions btcclient/tls.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package btcclient

import (
"io/ioutil"

"github.com/babylonchain/vigilante/config"
)

func readCAFile(cfg *config.BTCConfig) []byte {
// Read certificate file if TLS is not disabled.
var certs []byte
if !cfg.DisableClientTLS {
var err error
certs, err = ioutil.ReadFile(cfg.CAFile)
if err != nil {
log.Errorf("Cannot open CA file: %v", err)
// If there's an error reading the CA file, continue
// with nil certs and without the client connection.
certs = nil
}
} else {
log.Infof("Chain server RPC TLS is disabled")
}

return certs
}
30 changes: 30 additions & 0 deletions cmd/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package main

import (
"fmt"
"os"

"github.com/spf13/cobra"

"github.com/babylonchain/vigilante/cmd/reporter"
"github.com/babylonchain/vigilante/cmd/submitter"
)

// TODO: init log

func main() {
rootCmd := &cobra.Command{
Use: "vigilante",
Short: "Babylon vigilante",
}
rootCmd.AddCommand(reporter.GetCmd(), submitter.GetCmd())

if err := rootCmd.Execute(); err != nil {
switch e := err.(type) {
// TODO: dedicated error codes for vigilantes
default:
fmt.Print(e.Error())
os.Exit(1)
}
}
}
94 changes: 94 additions & 0 deletions cmd/reporter/reporter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
package reporter

import (
"github.com/babylonchain/vigilante/btcclient"
"github.com/babylonchain/vigilante/cmd/utils"
"github.com/babylonchain/vigilante/config"
vlog "github.com/babylonchain/vigilante/log"
"github.com/babylonchain/vigilante/metrics"
"github.com/babylonchain/vigilante/rpcserver"
"github.com/babylonchain/vigilante/vigilante"
"github.com/spf13/cobra"
)

var (
cfgFile = ""
log = vlog.Logger.WithField("module", "cmd")
)

// GetCmd returns the cli query commands for this module
func GetCmd() *cobra.Command {
// Group epoching queries under a subcommand
cmd := &cobra.Command{
Use: "reporter",
Short: "Vigilant reporter",
Run: cmdFunc,
}
addFlags(cmd)
return cmd
}

func addFlags(cmd *cobra.Command) {
cmd.Flags().StringVar(&cfgFile, "config", "", "config file")
}

func cmdFunc(cmd *cobra.Command, args []string) {
// get the config from the given file, the default file, or generate a default config
var err error
var cfg config.Config
if len(cfgFile) != 0 {
cfg, err = config.NewFromFile(cfgFile)
} else {
cfg, err = config.New()
}
if err != nil {
panic(err)
}

// create BTC client
btcClient, err := btcclient.New(&cfg.BTC)
if err != nil {
panic(err)
}
// create reporter
reporter, err := vigilante.NewReporter(&cfg.Reporter, btcClient)
if err != nil {
panic(err)
}
// crete RPC server
server, err := rpcserver.New(&cfg.GRPC, nil, reporter)
if err != nil {
panic(err)
}

// keep trying BTC client
btcClient.ConnectLoop()
// start reporter and sync
reporter.Start()
reporter.SynchronizeRPC(btcClient)
// start RPC server
server.Start()
// start Prometheus metrics server
metrics.Start()

// SIGINT handling stuff
utils.AddInterruptHandler(func() {
// TODO: Does this need to wait for the grpc server to finish up any requests?
log.Info("Stopping RPC server...")
server.Stop()
log.Info("RPC server shutdown")
})
utils.AddInterruptHandler(func() {
log.Info("Stopping reporter...")
reporter.Stop()
log.Info("Reporter shutdown")
})
utils.AddInterruptHandler(func() {
log.Info("Stopping BTC client...")
btcClient.Stop()
log.Info("BTC client shutdown")
})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to stop metrics?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seem no, as I did ctrl+c and all the goroutines will shut down gracefully, including metrics server

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the code again and found that you are right we need to stop the reporter/submitter here by reporter.Stop(), which also shuts btcClient. Not sure how to shut down metrics gracefully but I will look into it 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the graceful shutdown of submitter/reporter in the last PR. Do you think we can merge the metrics server into the RPC server here?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it should be merged. You mean if it should be served at, say, localhost:26001 or localhost:8081/metrics? I don't have any opinion on this, because in practice you can put the whole thing behind nginx and use it as a reverse proxy to do this. For sure Prometheus can usually be disabled by a config, but that can be done both ways.


<-utils.InterruptHandlersDone
log.Info("Shutdown complete")
}
94 changes: 94 additions & 0 deletions cmd/submitter/submitter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
package submitter

import (
"github.com/babylonchain/vigilante/btcclient"
"github.com/babylonchain/vigilante/cmd/utils"
"github.com/babylonchain/vigilante/config"
vlog "github.com/babylonchain/vigilante/log"
"github.com/babylonchain/vigilante/metrics"
"github.com/babylonchain/vigilante/rpcserver"
"github.com/babylonchain/vigilante/vigilante"
"github.com/spf13/cobra"
)

var (
cfgFile = ""
log = vlog.Logger.WithField("module", "cmd")
)

// GetCmd returns the cli query commands for this module
func GetCmd() *cobra.Command {
// Group epoching queries under a subcommand
cmd := &cobra.Command{
Use: "submitter",
Short: "Vigilant submitter",
Run: cmdFunc,
}
addFlags(cmd)
return cmd
}

func addFlags(cmd *cobra.Command) {
cmd.Flags().StringVar(&cfgFile, "config", "", "config file")
}

func cmdFunc(cmd *cobra.Command, args []string) {
// get the config from the given file, the default file, or generate a default config
var err error
var cfg config.Config
if len(cfgFile) != 0 {
cfg, err = config.NewFromFile(cfgFile)
} else {
cfg, err = config.New()
}
if err != nil {
panic(err)
}

// create BTC client
btcClient, err := btcclient.New(&cfg.BTC)
if err != nil {
panic(err)
}
// create submitter
submitter, err := vigilante.NewSubmitter(&cfg.Submitter, btcClient)
if err != nil {
panic(err)
}
// crete RPC server
server, err := rpcserver.New(&cfg.GRPC, submitter, nil)
if err != nil {
panic(err)
}

// keep trying BTC client
btcClient.ConnectLoop()
// start submitter and sync
submitter.Start()
submitter.SynchronizeRPC(btcClient)
// start RPC server
server.Start()
// start Prometheus metrics server
metrics.Start()

// SIGINT handling stuff
utils.AddInterruptHandler(func() {
// TODO: Does this need to wait for the grpc server to finish up any requests?
log.Info("Stopping RPC server...")
server.Stop()
log.Info("RPC server shutdown")
})
utils.AddInterruptHandler(func() {
log.Info("Stopping submitter...")
submitter.Stop()
log.Info("Submitter shutdown")
})
utils.AddInterruptHandler(func() {
log.Info("Stopping BTC client...")
btcClient.Stop()
log.Info("BTC client shutdown")
})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to stop metrics and submitter?


<-utils.InterruptHandlersDone
log.Info("Shutdown complete")
}
Loading