Skip to content
This repository has been archived by the owner on Aug 18, 2020. It is now read-only.

Jc/merge/release 2.0.0/develop #3982

Closed
wants to merge 147 commits into from
Closed

Conversation

jcmincke
Copy link
Contributor

Description

Merge from release/2.0.0 to develop.

I already solved the conflicts in wallet code. Please @KtorZ , double check it.

@iohk-devops : please solve the remaining conflicts:

Here is the list of the remaining conflicts:

both modified:   appveyor.yml
both added:      bors.toml
both modified:   default.nix
both modified:   docs/how-to/build-cardano-sl-and-daedalus-from-source-code.md
both modified:   nix/stack2nix.nix
both modified:   pkgs/default.nix
both modified:   release.nix
both modified:   stack.yaml

Linked issue

N/A

Type of change

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • 🛠 New feature (non-breaking change which adds functionality)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)
  • 🏭 Refactoring that does not change existing functionality but does improve things like code readability, structure etc
  • 🔨 New or improved tests for existing code
  • ⛑ git-flow chore (backport, hotfix, etc)

Developer checklist

  • I have read the style guide document, and my code follows the code style of this project.
  • If my code deals with exceptions, it follows the guidelines.
  • I have updated any documentation accordingly, if needed. Documentation changes can be reflected in opening a PR on cardanodocs.com, amending the inline Haddock comments, any relevant README file or one of the document listed in the docs directory.
  • CHANGELOG entry has been added and is linked to the correct PR on GitHub.

Testing checklist

  • I have added tests to cover my changes.
  • All new and existing tests passed.

QA Steps

Screenshots (if available)

How to merge

Send the message bors r+ to merge this PR. For more information, see
docs/how-to/bors.md.

kderme and others added 30 commits October 15, 2018 11:40
[CBR-466] fix address reported as unused during restoration
…de-600-x509

[CBR-470] Ensure 600 file permission on generated x509 certificates and keys
…g-GeneratedSecrets

[RCD-38] Fix "GeneratedSecrets missing" error
…ng-scribes

[CBR-477] Close scribes before terminating.
[DEVOPS-1099] make the wallet documentation optional, and default it to off
…-level

[RCD-42] log-configs/cluster.yaml: Include debug messages in json log
…/RCD-40

[release/2.0.0] [RCD-40] Address discrimination not enforced when sending transactions
…rs-origin

[DEVOPS-1121] explorer: add testnet URL to CORS origin whitelist
…elease

[DDW-480] Fix error message show user for x509 regeneration failures
[CSL-2526] update changes after covering remaining fees
….0.0

- List of placeholders generated from merge commits in the git log.
- Updated links in existing ChangeLog entries.
- Filled in ChangeLog entries for DEVOPS tickets.
(cherry picked from commit 6b6e22b)
rvl and others added 18 commits December 6, 2018 20:43
3934: [DEVOPS-1172] faucet: Use OptimizeForHighThroughput grouping policy r=rvl a=rvl

## Description

Fix faucet withdrawals which occasionally fail due to unavailable UTxO.

## Linked issue

https://iohk.myjetbrains.com/youtrack/issue/DEVOPS-1172



Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>
3936: [DEVOPS-1112] Staging application version r=cleverca22 a=rvl

## Description

Update the applicationVersion for the staging installers, but revert the part which changes the `lastKnownBlockVersion` for the staging cluster. This must be `0.2.0` not `0.1.0`.

## Linked issue

https://iohk.myjetbrains.com/youtrack/issue/DEVOPS-1112


Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>
Co-authored-by: Samuel Leathers <samuel.leathers@iohk.io>
This fix is actually two folds:

- It discards incoherent transactions fetched from the DB, if any,
  and shout a warning in the log. This is in order to make the system
  more resilient to conconcurrent calls while a wallet or account is
  being deleted (since metadata and accounts / wallets are stored in
  separated databases, we can't easily run both delete in a single
  transaction).

- It also deletes corresponding metadata when an account or a wallet
  is removed. This may cause extra damage? What if there are pending
  transactions when we delete the account or wallet.
The sqlite db is actually stored in a file and not a directory
3946: [DEVOPS-1112] Update ChangeLog for 2.0.0 r=jmitchell a=rvl



Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>
Co-authored-by: KtorZ <matthias.benkort@gmail.com>
3943: [CBR-495] Fix inconsistent metadata store after deletion r=jmitchell a=KtorZ

## Description

We store some information related to transaction in the metadata store (sqlite). However, when looking up transaction with metadata referring to wallet we don't know about, we fail with a not so friendly error "WalletNotFound" despite no wallet being given as part of the query.

This fix is actually two folds:

- It discards incoherent transactions fetched from the DB, if any,
  and shout a warning in the log. This is in order to make the system
  more resilient to conconcurrent calls while a wallet or account is
  being deleted (since metadata and accounts / wallets are stored in
  separated databases, we can't easily run both delete in a single
  transaction).

- It also deletes corresponding metadata when an account or a wallet
  is removed. This may cause extra damage? What if there are pending
  transactions when we delete the account or wallet.

## Linked issue

[CBR-495](https://iohk.myjetbrains.com/youtrack/issue/CBR-495)



3947: [CBR-496] Fix --rebuild-db not rebuilding sqlite r=jmitchell a=KtorZ

## Description

<!--- A brief description of this PR and the problem is trying to solve -->

Turns out we aren't actually removing the sqlite database when passing --rebuild-db becaue the SQLite db is actually stored in a file and not a directory. 

## Linked issue

<!--- Put here the relevant issue from YouTrack -->

[[CBR-496]](https://iohk.myjetbrains.com/youtrack/issue/CO-445)



Co-authored-by: KtorZ <matthias.benkort@gmail.com>
Co-authored-by: kderme <kostas.dermentzis@iohk.io>
…-version-staging

[DEVOPS-1112] Bump staging applicationVersion 17 → 18
# Conflicts:
#	appveyor.yml
#	bors.toml
#	default.nix
#	docs/how-to/build-cardano-sl-and-daedalus-from-source-code.md
#	nix/stack2nix.nix
#	pkgs/default.nix
#	release.nix
#	stack.yaml
#	wallet-new/cardano-sl-wallet-new.cabal
#	wallet-new/src/Cardano/Wallet/API/Internal/LegacyHandlers.hs
#	wallet-new/src/Cardano/Wallet/Kernel/CoinSelection/Generic/Fees.hs
#	wallet-new/src/Cardano/Wallet/Kernel/DB/TxMeta.hs
#	wallet-new/src/Cardano/Wallet/Kernel/Migration.hs
#	wallet-new/src/Cardano/Wallet/Server/LegacyPlugins.hs
#	wallet/src/Cardano/Wallet/API/Internal.hs
#	wallet/src/Cardano/Wallet/API/Internal/Handlers.hs
#	wallet/src/Cardano/Wallet/Kernel/PrefilterTx.hs
#	wallet/src/Cardano/Wallet/WalletLayer.hs
#	wallet/src/Pos/Wallet/Web/State/Storage.hs
#	wallet/test/unit/Test/Spec/GetTransactions.hs
#	wallet/test/unit/Test/Spec/TxMetaStorage.hs

Initial status:
Changes to be committed:

	modified:   .buildkite/pipeline.yml
	new file:   .buildkite/windows.ps1
	modified:   db/cardano-sl-db.cabal
	modified:   db/src/Pos/DB/Block/Epoch.hs
	modified:   db/src/Pos/DB/Rocks/Functions.hs
	modified:   lib/src/Pos/Diffusion/Full/Block.hs
	modified:   log-configs/connect-to-cluster.yaml
	modified:   tools/cardano-sl-tools.cabal
	modified:   tools/src/launcher/Main.hs

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)

	both modified:   appveyor.yml
	both added:      bors.toml
	both modified:   default.nix
	both modified:   docs/how-to/build-cardano-sl-and-daedalus-from-source-code.md
	both modified:   nix/stack2nix.nix
	both modified:   pkgs/default.nix
	both modified:   release.nix
	both modified:   stack.yaml
	deleted by us:   wallet-new/cardano-sl-wallet-new.cabal
	deleted by us:   wallet-new/src/Cardano/Wallet/API/Internal/LegacyHandlers.hs
	deleted by us:   wallet-new/src/Cardano/Wallet/Kernel/CoinSelection/Generic/Fees.hs
	deleted by us:   wallet-new/src/Cardano/Wallet/Kernel/DB/TxMeta.hs
	deleted by us:   wallet-new/src/Cardano/Wallet/Kernel/Migration.hs
	deleted by us:   wallet-new/src/Cardano/Wallet/Server/LegacyPlugins.hs
	both modified:   wallet/src/Cardano/Wallet/API/Internal.hs
	both modified:   wallet/src/Cardano/Wallet/API/Internal/Handlers.hs
	both modified:   wallet/src/Cardano/Wallet/Kernel/PrefilterTx.hs
	both modified:   wallet/src/Cardano/Wallet/WalletLayer.hs
	deleted by us:   wallet/src/Pos/Wallet/Web/State/Storage.hs
	both modified:   wallet/test/unit/Test/Spec/GetTransactions.hs
	both modified:   wallet/test/unit/Test/Spec/TxMetaStorage.hs
default.nix Outdated
=======
cardano-sl-wallet
cardano-sl-wallet-new
cardano-sl-wallet-new-static
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no more "wallet-new"

- `cardano-wallet-static`:
=======
- `cardano-sl-wallet-new-static`:
>>>>>>> release/2.0.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, no more "wallet-new"

@erikd erikd self-requested a review January 2, 2019 00:17
Copy link
Member

@erikd erikd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we doing this?

This is a large PR which is difficult to review and contains lots of completely unrelated changes.

If there are specific commits on the release-2.0.0 branch that we want on develop then we should individually cherry pick those commits. Also we should avoid as much as possible doing development on release branches.

@rvl
Copy link
Contributor

rvl commented Jan 4, 2019

I resolved the remaining conflicts as requested.

If you view the diff of a merge commit in gitk rather than github then it's much clearer.

There wasn't any development done on this release branch. It was only bug fixes and build fixes.

Copy link
Contributor

@KtorZ KtorZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are missing:

# After some investigation, it was discovered that this was because 'rocksdb.dll' has to be located in this folder as well, or else the test executable doesn't work.
copy rocksdb.dll node
copy rocksdb.dll lib
copy rocksdb.dll wallet-new
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect. It should be wallet


Write-Output "+++ Stack build and test"

stack.exe --dump-logs install cardano-sl cardano-sl-tools cardano-sl-wallet-new -j 3 --no-terminal --local-bin-path $env:WORK_DIR --no-haddock-deps --flag cardano-sl-core:-asserts --flag cardano-sl-tools:for-installer --extra-include-dirs="Z:\OpenSSL-Win64-v102\include" --extra-lib-dirs="Z:\OpenSSL-Win64-v102" --extra-include-dirs="Z:\xz_extracted\include" --extra-lib-dirs="Z:\xz_extracted\bin_x86-64" --extra-include-dirs="$env:WORK_DIR\rocksdb\include" --extra-lib-dirs="$env:WORK_DIR"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cardano-sl-wallet-new should be cardano-wallet

, filepath: "node"
, logsafety: SecretLogLevel
, severity: Info+
, backend: FileTextBE }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeFile (toString runnerPath) $ unlines
[ "TaskKill /IM cardano-launcher.exe /F"
[ "TaskKill /PID "<>show selfPid<>" /F"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Those are the missing changes from #3926

@@ -156,6 +156,8 @@ executable cardano-launcher
, yaml
if !os(windows)
build-depends: unix
else
build-depends: Win32
default-language: Haskell2010
ghc-options: -threaded
-Wall
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Those are the missing changes from #3926

@@ -47,6 +47,7 @@ import Data.Binary (decode, encode)
import qualified Data.ByteString.Char8 as BS
import qualified Data.ByteString.Lazy as LBS
import Data.Either (partitionEithers)
import Data.List.Extra (chunksOf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

=> Handle -> (Int, SlotIndexHash) -> m (Either ConsolidateError SlotIndexLength)
consolidate hdl (indx, SlotIndexHash lsi hh) = do
when (indx `mod` 1000 == 0) $
sleepSeconds 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -141,6 +141,7 @@ library
, ekg-core
, ether >= 0.5
, exceptions
, extra
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

command: 'PowerShell.exe -executionpolicy remotesigned -File .\.buildkite\windows.ps1'
agents:
system: x86_64-win2016

Copy link
Contributor

@KtorZ KtorZ Jan 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 These are the missing changes from #3880, #3912, #3913 & #3927


$daedaluszip = "$env:BUILDKITE_COMMIT.zip"
7z.exe a $daedaluszip .\daedalus\*
buildkite-agent artifact upload $daedaluszip
Copy link
Contributor

@KtorZ KtorZ Jan 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 These are the missing changes from #3880, #3912, #3913 & #3927

@KtorZ
Copy link
Contributor

KtorZ commented Jan 4, 2019

Here below, the complete list of PR made against release/2.0.0 since the cut-off and, if any and applicable, their equivalent on develop.

YT Ticket develop release/2.0.0
?? #3972 #3846
?? #3888
?? #3902 #3892
?? #3722 #3909
?? N/A (newer version on develop) #3924
CBR-466 #3804 #3742
CBR-470 #3805 #3773
CBR-475 #3873 #3870
CBR-477 #3807 #3810
CBR-495 #146 #3943
CBR-496 #138 #3947
CSL-2526 #3815 #3833
DDW-480 #3834 #3838, #3835
DEVOPS-1094 #3972 #3817
DEVOPS-1099 #3825 #3771
DEVOPS-1112 #3972 #3962, #3936, #3946, #3919, #3879
DEVOPS-1113 PENDING #3982 #3880, #3912, #3913, #3927
DEVOPS-1121 #3972 #3831
DEVOPS-1126 #3852
DEVOPS-1130 #3849 #3848
DEVOPS-1134 #3979 #3856
DEVOPS-1141 #3972 #3867
DEVOPS-1143 N/A (newer version on develop) #3889, #3907
DEVOPS-1153 N/A (legacy data-layer) #3911
DEVOPS-1172 #3972 #3934
RCD-37 #3972 #3811
RCD-38 #3813 #3812
RCD-40 #3823 #3819
RCD-42 #3972 #3816
RCD-43 #3973 #3931, #3920
RCD-45, RCD-44 #3875 #3861
RCD-46 #3875 #3855
RCD-47 N/A (feature no-longer exists) #3876
RCD-48 #3972 #3883
RCD-49 #77 #3886
RCD-52 PENDING #3982 #3926

writeFile (toString runnerPath) $ unlines
[ "TaskKill /PID "<>show selfPid<>" /F"
[ "TaskKill /IM cardano-launcher.exe /F"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@disassembler
Copy link
Contributor

Closing in favor of manual cherry picks in https://iohk.myjetbrains.com/youtrack/issue/DEVOPS-1211

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.