Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serializer benchmarks #23

Merged
merged 10 commits into from
Apr 18, 2018
Merged

Conversation

adamsitnik
Copy link
Member

This PR adds the serializer benchmarks.

The idea is simple:

  1. Project Definitions defines benchmarks and targets .NET Standard 2.0
  2. Project Runner references Definitions and targets multiple frameworks (.NET 4.6.1, Core 2.0, Core 2.1) and allows to run them using BenchmarkDotNet

Todo (why it's WIP):

  • console args support (sth like --coreclr Path --corefx Path)
  • Readme

@adamsitnik adamsitnik requested a review from jorive April 6, 2018 12:37
…ting 32/64 bit benchmarks, emit full pdb info to get disassembler support
@@ -0,0 +1,49 @@
#if SGEN
// the new SGEN tool fails to load some of the dependencies, so we need to replace the problematic dependencies for this particular build configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this file for?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to benchmark the XmlSerializer with SGEN enabled to measure the gains. But the new SGEN tool fails to load some of the dependencies, so I needed to replace the problematic dependencies for the "SGEN" build configuration.

so it is something like

#if SGEN
using fake dependencies;
#else
using real dependencies;
#endif

See https://github.com/dotnet/corefx/issues/27281#issuecomment-367449130 for more

The problem is that it does not work yet, so maybe I should just remove it to keep it clean?

@jorive
Copy link
Member

jorive commented Apr 6, 2018

I think src/benchmarks/ should be src/serializer/ as we will be adding more benchmarks.

@adamsitnik
Copy link
Member Author

I think src/benchmarks/ should be src/serializer/ as we will be adding more benchmarks.

I was not sure about how to name it and where to add it.

There is src\docker so I decided to create a new folder, for benchmarks only. This is how I ended up with src\benchmarks\Definitions\Serializers

@adamsitnik
Copy link
Member Author

Maybe I should keep it in src\benchmarks\Serializers ?

@jorive
Copy link
Member

jorive commented Apr 6, 2018

Are we planning on using the same solution/project for all benchmarks?

@adamsitnik
Copy link
Member Author

Yes, I was thinking about having a single project with benchmarks, then every benchmark "category" having it's own folder. Do you like this idea?

@jorive
Copy link
Member

jorive commented Apr 6, 2018

Yes, that could work.

Copy link
Member

@jorive jorive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jorive
Copy link
Member

jorive commented Apr 6, 2018

We should start thinking about CI integration tests.

Copy link
Member

@jorive jorive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@adamsitnik adamsitnik changed the title [WIP] Serializer benchmarks Serializer benchmarks Apr 17, 2018
@adamsitnik
Copy link
Member Author

@jorive I have added the docs (it took me more time than expected). Could you check my most recent commit?

* [JSON.NET](https://github.com/JamesNK/Newtonsoft.Json) `11.0.1`
* [Utf8Json](https://github.com/neuecc/Utf8Json) `1.3.7`
* Binary
* BinaryFormatter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use this:
BinaryFormatter version

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe same for the others?

@jorive
Copy link
Member

jorive commented Apr 17, 2018

@adamsitnik LGTM

@adamsitnik adamsitnik merged commit 5feed32 into dotnet:master Apr 18, 2018
@adamsitnik adamsitnik deleted the serializerBenchmarks branch October 17, 2018 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants