Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update #2

Merged
merged 57 commits into from
Jun 29, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
95e474b
preload and not bloking
yeion7 Jun 23, 2018
d6f1dbf
fix: use ES5 syntax
Jun 24, 2018
feb0193
fix: remove CSS loading script
Jun 24, 2018
f41eb2a
fix: move async CSS load to gatsby-browser.js
jlengstorf Jun 24, 2018
61d52af
[gatsby-source-contentful] delay creating cache directory (#6121)
pieh Jun 25, 2018
ee393e7
fix migration docs link in README (#6119)
ds300 Jun 25, 2018
cb65a7a
fix typo (#6120)
manpenaloza Jun 25, 2018
b80d6eb
Fix casing of JavaScript in all docs (#6115)
detj Jun 25, 2018
bb1d429
Export graphql TypeScript definition from gatsby package (#6096)
dannywils Jun 25, 2018
e117179
Treat integers longer than 32 bit as floats. (#6082)
MichaelZoerner Jun 25, 2018
cdfbb95
create public page renderer and export correct version (dev / prod) d…
pieh Jun 25, 2018
3e1c91d
[gatsby-source-mongodb] added support for extra connection string par…
lcostea Jun 25, 2018
60ab254
add gatsby-starter-skeleton (#6141)
Undistraction Jun 25, 2018
2588a2b
Update plugins.md (#6099)
calcsam Jun 25, 2018
2e80bac
update yarn.lock
pieh Jun 25, 2018
2477243
chore(release): Publish
pieh Jun 25, 2018
f466c7c
Make getElementById Conditional (#6145)
jigz Jun 25, 2018
b252aba
[v2] migrate gatsby-plugin-feed to use new Link (#6147)
daydream05 Jun 25, 2018
445f5ff
[gatsby-source-contentful] pass host to client init (#6149)
pieh Jun 25, 2018
0d3d1bb
chore(release): Publish
pieh Jun 25, 2018
bd30efa
[v2] Fix doc not centered (#6146)
Jun 25, 2018
9166295
Move store assignment out of component. (#6150)
MichaelZoerner Jun 25, 2018
7fd1898
fix displaying of graphql resolver errors (#6142)
pieh Jun 26, 2018
26582d3
Tidy up the remark example (#6160)
m-allanson Jun 26, 2018
0591eb3
Fix old query results showing up in develop (#6159)
m-allanson Jun 26, 2018
5150616
delay page transition until we have all resources (#6158)
pieh Jun 26, 2018
3f35009
[www] Gatsby/rebeccapurple banner (#6161)
fk Jun 26, 2018
2512eef
chore(release): Publish
pieh Jun 26, 2018
2027091
Fix link to component from gatsbyjs.org fixes #6162
KyleAMathews Jun 27, 2018
3d69586
Merge pull request #6178 from gatsbyjs/KyleAMathews-patch-2
KyleAMathews Jun 27, 2018
c559185
Update gatsby-on-windows to include mention that Node x64 is a requir…
KyleAMathews Jun 27, 2018
a0f0fc6
Merge pull request #6179 from gatsbyjs/KyleAMathews-patch-2
KyleAMathews Jun 27, 2018
7d11cee
fixed typo in gatsby-remark-prismjs README.md (#6165)
drenther Jun 27, 2018
75937d7
Fix typo (#6166)
giraffesyo Jun 27, 2018
3396f38
Update node-interface.md (#6171)
janosh Jun 27, 2018
0e5aa1b
Increase author link visibility & link avatars in blog posts + previe…
rdela Jun 27, 2018
7190fe1
fix: use the language loader for imports not postcss (#6173)
jquense Jun 27, 2018
5ed1efc
Update sites.yml (#6164)
ooloth Jun 27, 2018
5ca3f2f
chore(release): Publish
pieh Jun 27, 2018
3809d03
[WIP][www] Sidebars à la reactjs.org (accordions, scrollspy) (#6153)
fk Jun 27, 2018
d73610a
Card sort blogpost (#6152)
shannonbux Jun 27, 2018
4030f99
Adding NYC Pride 2019 site to showcase sites.yml (#6187)
lesliecdubs Jun 27, 2018
3a05c53
Clarify what site the card sort was for
KyleAMathews Jun 28, 2018
b9f0f21
[www] Fix package.json name, description, version (#6198)
fk Jun 28, 2018
5dfd4d3
[www] Import graphql from Gatsby (#6196)
fk Jun 28, 2018
58571be
docs(gatsby-image): Minor cleanup (#6185)
btmills Jun 28, 2018
386db39
stop builds on errors in built-html step (#6188)
pieh Jun 28, 2018
261286a
docs: add typography changes to migration guide (#6151)
NickyMeuleman Jun 28, 2018
1b457b6
[v2][tutorial] Add importing graphql function (#6092)
Jun 28, 2018
efa43fb
Add "group" to graphql reference
LekoArts Jun 28, 2018
02a0f40
Fix Showcase header `position:sticky` `top` (#6203)
fk Jun 28, 2018
74909d0
new blog post (#6155)
calcsam Jun 28, 2018
fd6cc1a
[gatsby-source-medium] fix next version (#6205)
pieh Jun 28, 2018
6e52c82
add migrating doc with update gatsby related deps
curiouslychase Jun 28, 2018
eeef674
Update gatsby-source-hacker-news and the hn example to Gatsby v2 (#6214)
m-allanson Jun 28, 2018
6986fb5
Update migrating-from-v1-to-v2.md
KyleAMathews Jun 28, 2018
dd91b8d
Merge pull request #6213 from chaseadamsio/patch-1
KyleAMathews Jun 28, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ Websites built with Gatsby:

**[View the docs on gatsbyjs.org](https://www.gatsbyjs.org/docs/)**

[Migrating from v1 to v2?](https://www.gatsbyjs.org/docs/migrating-from-v1-to-v2/)
[Migrating from v1 to v2?](https://next.gatsbyjs.org/docs/migrating-from-v1-to-v2/)

[Migrating from v0 to v1?](https://www.gatsbyjs.org/docs/migrating-from-v0-to-v1/)

Expand Down
4 changes: 2 additions & 2 deletions docs/blog/2017-09-18-gatsby-modern-static-generation/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ need to maintain a runtime, a database, application code, or complex optimized
web servers.

But the downside is you have to keep separate template files that will
eventually rendered as HTML on the browser, and Javascript file that will
eventually rendered as HTML on the browser, and JavaScript file that will
eventually manipulate that HTML again on the browser (and CSS assets too).

## Enter the new world order
Expand Down Expand Up @@ -86,7 +86,7 @@ On your end, the development flow looks:
- During development, write code like a boss (hot reloading, modularized code,
webpack plugins, etc...)
- Use React.js Server Side Rendering API to convert this code to static HTML
content, and Javascript code, on your server
content, and JavaScript code, on your server

It’s really that easy? Well, sort of.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ features. I haven’t heard any complaints from my co-workers.
Another “problem” with static sites is the lack of out-of-the-box site search.
Most search implementations occur between the server and the database. As a doc
site, users typically expect solid search functionality. There are a few
frontend only javascript search libraries (like [lunr.js](https://lunrjs.com/))
frontend only JavaScript search libraries (like [lunr.js](https://lunrjs.com/))
that take a search query and parse a pre-built JSON index of your content.

I could have created this index by tying into the `onPostBuild` Gatsby API. This
Expand Down
2 changes: 1 addition & 1 deletion docs/blog/2018-02-16-bright-future-for-the-web/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ _This article was originally published on
[Medium](https://medium.com/@ryanwiemer/gatsby-and-the-jam-stack-91e31508f364)
on January 31, 2018._

Recently I relaunched my wife’s photography portfolio, [KNW Photography](https://www.knw.io/), using a combination of Gatsby, [Contentful](https://www.contentful.com/) and [Netlify](https://www.netlify.com/). This particular group of tools represents a new and exciting web development architecture known as the [JAMstack](https://jamstack.org/) (Javascript, APIs and Markup). In this post I will be sharing my personal thoughts on each of these new tools and why together they represent the “holy grail” of the static website world.
Recently I relaunched my wife’s photography portfolio, [KNW Photography](https://www.knw.io/), using a combination of Gatsby, [Contentful](https://www.contentful.com/) and [Netlify](https://www.netlify.com/). This particular group of tools represents a new and exciting web development architecture known as the [JAMstack](https://jamstack.org/) (JavaScript, APIs and Markup). In this post I will be sharing my personal thoughts on each of these new tools and why together they represent the “holy grail” of the static website world.

## Why Other Static Site Generators Didn’t Work For Me

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ Long story short, I decided to get rid of WordPress and replace it with a much m

JAM is an acronym for **J**avascript **A**PI **M**arkup. What does it mean though? It is not a framework or a specific technology. It is more of an architectural approach. A modern way web content is built, managed and distributed.

Back in the day, the web was just static HTML pages, maybe with some CSS and Javascript on the top. Perfect for static content. However, usually, you need your data to be more dynamic. Instead of static HTML sitting on the server served directly, HTML pages were built on-the-fly when a client requested them. The data dynamically provided from a database. While this is very flexible, it comes at a cost. It's slow. Way slower than just a static site. The application server executes code on demand, needs to contact a database, usually on a different machine, build the page and send it back. The scalability is hard and expensive. When the traffic rises, its more complicated to scale the solution. We're talking load balancing, session replication, distributed caching not to mention difficult RDS scalability. And it's vulnerable. With so many moving parts, the attack surface is extensive. Ranging from Cross-Site-Scripting, SQL Injection to exploiting [security vulnerabilities in your libraries](https://www.vojtechruzicka.com/snyk-detecting-dependencies-with-known-vulnerabilities/).
Back in the day, the web was just static HTML pages, maybe with some CSS and JavaScript on the top. Perfect for static content. However, usually, you need your data to be more dynamic. Instead of static HTML sitting on the server served directly, HTML pages were built on-the-fly when a client requested them. The data dynamically provided from a database. While this is very flexible, it comes at a cost. It's slow. Way slower than just a static site. The application server executes code on demand, needs to contact a database, usually on a different machine, build the page and send it back. The scalability is hard and expensive. When the traffic rises, its more complicated to scale the solution. We're talking load balancing, session replication, distributed caching not to mention difficult RDS scalability. And it's vulnerable. With so many moving parts, the attack surface is extensive. Ranging from Cross-Site-Scripting, SQL Injection to exploiting [security vulnerabilities in your libraries](https://www.vojtechruzicka.com/snyk-detecting-dependencies-with-known-vulnerabilities/).

When you just have static HTML, CSS and Javascript files, it's much harder to compromise your site unless someone gains complete access to your server. Since the content does not change, it is easy to cache. Instead of having just one central server or just a few of them in an expensive and complicated solution, you can use Content Delivery Network to distribute your site. This way you'll have your availability covered and CDN redistributing your files across the globe. Users from New York will be served as fast as users from Tokyo. And if your traffic skyrockets, it is still not a problem. Scalability is easy and cheap if you don't have to manage synchronization of state.
When you just have static HTML, CSS and JavaScript files, it's much harder to compromise your site unless someone gains complete access to your server. Since the content does not change, it is easy to cache. Instead of having just one central server or just a few of them in an expensive and complicated solution, you can use Content Delivery Network to distribute your site. This way you'll have your availability covered and CDN redistributing your files across the globe. Users from New York will be served as fast as users from Tokyo. And if your traffic skyrockets, it is still not a problem. Scalability is easy and cheap if you don't have to manage synchronization of state.

![jam stack](./jamstack.png)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,6 @@ Contentful ensures your website and its users can access your content quickly an

That’s a quick overview of how to get your website up and running from scratch using Contentful and GatsbyJS.

Contentful's platform offers speed, flexibility, and ease of integration with your code, supporting any stack you use with SDKs for common languages like [Javascript](https://www.contentful.com/developers/docs/javascript/sdks/), [Python](https://www.contentful.com/developers/docs/python/sdks/), and [PHP](https://www.contentful.com/developers/docs/php/sdks/). Unlike a CMS, Contentful provides you with separation between content and presentation, allowing you to focus on developing your website and leave content delivery to Contentful.
Contentful's platform offers speed, flexibility, and ease of integration with your code, supporting any stack you use with SDKs for common languages like [JavaScript](https://www.contentful.com/developers/docs/javascript/sdks/), [Python](https://www.contentful.com/developers/docs/python/sdks/), and [PHP](https://www.contentful.com/developers/docs/php/sdks/). Unlike a CMS, Contentful provides you with separation between content and presentation, allowing you to focus on developing your website and leave content delivery to Contentful.

If you're curious to learn more about Contentful, you can check out the Contentful [Developer Center](https://www.contentful.com/developers/docs/) to learn more on how the technology works, or head over to the [Guides and tutorials](https://www.contentful.com/guides/) section.
4 changes: 2 additions & 2 deletions docs/blog/2018-06-08-life-after-layouts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ In short, the decision to remove layouts was part of an effort to reduce unneces

### What was the unnecessary complexity?

Because layouts in V1 were applied under the hood, it made it _really hard_ to pass data between the layout the pages it wrapped.
Because layouts in V1 were applied under the hood, it made it _really hard_ to pass data between the layout and the pages it wrapped.

In short, layouts in V1 broke React's compositional model, and that made things unnecessarily hard to implement. By going back to the standard React way of importing components and using them directly, we're able to easily pass data between the layout and pages without any confusing, complicated, or otherwise unwieldy workarounds.

Expand All @@ -93,7 +93,7 @@ In V1, if something breaks on your page, you'd likely go to the page, look throu

Because Gatsby is doing things in the background, out of sight of the developers building the site, we're creating a mystery: where the hell did this bug come from if all the code I can see doesn't have the bug?

By removing the magic and making layouts an explicity dependency in V2, that same bug would be much easier to spot because there's a clear link between our page code and the layout.
By removing the magic and making layouts an explicit dependency in V2, that same bug would be much easier to spot because there's a clear link between our page code and the layout.

### Why is this better?

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ If you came across [Gatsby](https://github.com/gatsbyjs/gatsby) you will notice

![gatsby-logo](https://images.ctfassets.net/4x6byznv2pet/4OW1X9ex1mImko8G4w4WAK/a16fceab310b718c7f375a760c4e1e16/logo-gatsby-0603eb9dd6bdfec9599dbc7590f891be-347ea.jpg)

Gatsby is a blazing fast static site generator for [React](https://github.com/facebook/react). Actually, it is more than that. Think of it as a PWA (Progressive Web App) framework with best practices backed in. For example: you get code and data splitting out-of-the-box.
Gatsby is a blazing fast static site generator for [React](https://github.com/facebook/react). Actually, it is more than that. Think of it as a PWA (Progressive Web App) framework with best practices baked in. For example: you get code and data splitting out-of-the-box.

## Why Move to Gatsby?

Expand Down Expand Up @@ -126,7 +126,7 @@ class BlogPost extends Component {
componentDidMount() {
getBlogPost(this.props.match.slug)
.then((data) => this.setState({ data }))
.catch((error) => this.setState({ state: 'error' }))
.catch((error) => this.setState({ status: 'error' }))
}
render() {
if (!this.state.status === 'error') {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
title: Investigating Build Performance At Scale
date: "2018-06-24"
author: "Sam Bhagwat"
tags: ["scalability", "builds"]
---
When website teams are deciding whether to migrate an existing site over to Gatsby, one consideration is how long it takes to rebuild each site when code or content changes.

In order to help teams evaluate this, we’re publishing a set of build time benchmarks.

# Performance Considerations

Generally speaking, Gatsby build times consist of two factors -- O(1) constant-time operations, such as connecting to remote APIs, and O(n) linear-time operations, such as processing markdown pages, transforming images, and so on.

On smaller sites, O(1) constant time operations take up most of the time “cost” of building a Gatsby site. On larger sites, O(n) linear-time operations predominate. Generally, larger sites tend to be CPU-bound rather than I/O-bound, so a more powerful server, VM, or container will finish quicker.

The three most memory-intensive operations that scale with site size are **processing markdown**, **creating pages**, and **processing images with gatsby-image**.

# Benchmark Parameters

_Site tests_

In order to measure the “build performance cost” of each of these three factors, we’ve benchmarked Gatsby build times for two different sites, one image-heavy and one markdown-page heavy.

The first site is [FreeCodeCamp’s Guide](https://github.com/freeCodeCamp/guide), one of the largest open-source Gatsby sites with around 3,000 pages. The second site is the using-gatsby-image official example [copied into its own repo](https://github.com/calcsam/gatsby-image-performance-benchmarking). We’ve run two different versions of this: first, with around 25 images and second, with around 250 images. Both sites are running on Gatsby v2.

Note that **using gatsby-image is not a requirement to use Gatsby**. If you have enough images that image-processing-driven build times is impeding project goals or team workflows, consider simply removing (or not implementing) gatsby-image.

_Machine types_

Tests were run on three different machines; on a local development machine (2015 Macbook Pro), on an AWS t2 micro free tier instance, and an AWS memory-optimized m4 instance.

_Cold start vs warm start_

Tests were run both with (“warm start) and without (“cold start”) a pre-existing, local Gatsby cache.

In most cases when code changes and all cases when content changes, Gatsby will preserve the cache from your previous build, giving you a warm start. The cache is only invalidated in rare instances, for example when the plugin changes.

Whether you’re able to access the cache from previous builds depends on your CI setup, but many CI providers offer the option of preserving the previous build cache.

# Results

For cold start builds, each additional markdown page adds around **0.17 seconds**, while each additional image processed with gatsby-image adds **between 1.5 and 2.1 seconds** to fresh Gatsby builds.

For warm start builds, each additional markdown page adds **around 0.07 seconds**, while additional images processed with gatsby-image are free.

Full results are [linked](https://docs.google.com/spreadsheets/d/1ki5PwVTnIyycsk800DSWIA72UAr1k1DUJnKCf_lWz4c/edit#gid=0), along with our [build script](https://gist.github.com/calcsam/4aa066a46d74b6713c053a6adc0e0f76).

# Next steps

Gatsby users report that Gatsby builds fail due to running against hard Node.js process memory limits somewhere around 10,000 to 15,000 pages on v1.

Since v2 has improved memory usage, we want to update this number, as well as further quantify the boundary so that teams considering migrating larger sites to Gatsby have more information for their decision-making.

We also want to benchmark v2 build performance against v1 build performance for various sites.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading