Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DocPad is regenerating more files than it needs to #785

Closed
balupton opened this issue Jan 27, 2014 · 13 comments
Closed

DocPad is regenerating more files than it needs to #785

balupton opened this issue Jan 27, 2014 · 13 comments
Labels
1 - confirmed affects core affects watching contributions welcome difficulty days type defect

Comments

@balupton
Copy link
Member

Myself, @stongo, and @RobLoach have all encountered this with the recent releases. At some point, DocPad has started generating more files than it needs to. This is a regression, as DocPad didn't use to do this. We need to look into this.
@dimitarkolev
Copy link

I believe that Issue #770 is related to this one.

@dimitarkolev dimitarkolev mentioned this issue Jan 27, 2014
Closed
@stongo
Copy link

stongo commented Jan 27, 2014

Tracked down the version where this starts happening. The "standalone"
meta data property stopped working in 6.59.x

On 2014-01-26 9:00 PM, Benjamin Arthur Lupton wrote:

Myself, @stongo https://github.com/stongo, and @RobLoach
https://github.com/RobLoach have all encountered this with the
recent releases. At some point, DocPad has started generating more
files than it needs to. This is a regression, as DocPad didn't use to
do this. We need to look into this.

—
Reply to this email directly or view it on GitHub
#785.

@stongo
Copy link

stongo commented Jan 27, 2014

6.59.1 to be specific

@stongo
Copy link

stongo commented Jan 27, 2014

errr... 6.59.0 rather
tested using twitter-bootstrap-jade.docpad

@balupton
Copy link
Member Author

Hrmm, the changelog for v6.59.0 — v6.58.2...v6.59.0 — doesn't seem to show why that would be the cause for any change, strange.

@stongo
Copy link

stongo commented Jan 27, 2014

Update to watchr stands out as a possible cause

On 2014-01-27 4:36 PM, Benjamin Arthur Lupton wrote:

Hrmm, the changelog for v6.59.0 — v6.58.2...v6.59.0
v6.58.2...v6.59.0 — doesn't
seem to show why that would be the cause for any change, strange.

—
Reply to this email directly or view it on GitHub
#785 (comment).

@balupton
Copy link
Member Author

For regenerations on interconnect:

  • DocPad v6.59.0: info: Generated 23/24 files in 3.749 seconds
  • DocPad v6.58.2: info: Generated 23/24 files in 3.952 seconds
  • DocPad v6.57.3: info: Generated 7/24 files in 2.728 seconds
  • DocPad v6.56.0: info: Generated 7/24 files in 2.692 seconds
  • DocPad v6.55.9: info: Generated 7/24 files in 4.38 seconds
  • DocPad v6.54.10: info: Generated 7/24 files in 4.38 seconds

At least for interconnect, this seems to be the changes that broke it:

v6.57.3...v6.58.1

@stongo
Copy link

stongo commented Jan 27, 2014

I was tracking the "standalone" meta attribute. It still works in
v6.58.2 and not at all in higher versions

On 2014-01-27 5:14 PM, Benjamin Arthur Lupton wrote:

DocPad v6.58.2: |info: Generated 23/24 files in 3.952 seconds|

balupton added a commit that referenced this issue Jan 27, 2014
@balupton
Found an incorrect flag related to generations. Ref #785
f99f053
@balupton
Copy link
Member Author

@stongo okay cool, will look into that too

@balupton
Copy link
Member Author

It seems it's something busted with the mtimes. They are always greater than the last generate time...

info: The action completed successfully
    → [2014-01-27 17:53:23.268] [/Users/balupton/Projects/docpad/out/lib/interfaces/console.js:161] [completeAction]
debug: Watch triggered on: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app
    → [2014-01-27 17:53:32.805] [events.js:98] [_Class.EventEmitter.emit]
debug: Watch followed through on: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app
    → [2014-01-27 17:53:32.811] [events.js:98] [_Class.EventEmitter.emit]
debug: Watch triggered on: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:32.993] [events.js:98] [_Class.EventEmitter.emit]
debug: Watch followed through on: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.001] [events.js:98] [_Class.EventEmitter.emit]
debug: Determined update: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.007] [events.js:98] [_Class.EventEmitter.emit]
info: Change detected at 17:53:33 update /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.012] [/Users/balupton/Projects/docpad/out/lib/docpad.js:3572] [_Class.changeHandler]
debug: Detect the file: document: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.017] [/Users/balupton/Projects/docpad/out/lib/docpad.js:2265] [DocumentModel.<anonymous>]
debug: Stat the file and cache the result: document: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.023] [/Users/balupton/Projects/docpad/out/lib/docpad.js:2265] [DocumentModel.<anonymous>]
debug: Read the file and cache the result: document: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.028] [/Users/balupton/Projects/docpad/out/lib/docpad.js:2265] [DocumentModel.<anonymous>]
debug: Load -> Parse: document: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.034] [/Users/balupton/Projects/docpad/out/lib/docpad.js:2265] [DocumentModel.<anonymous>]
debug: Parse -> Normalize: document: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.041] [/Users/balupton/Projects/docpad/out/lib/docpad.js:2265] [DocumentModel.<anonymous>]
debug: Normalize -> Contextualize: document: /Users/balupton/Projects/interconnect/src/documents/elements/rtc-app/rtc-app.html
    → [2014-01-27 17:53:33.049] [/Users/balupton/Projects/docpad/out/lib/docpad.js:2265] [DocumentModel.<anonymous>]
info: Regenerating at 17:53:33
    → [2014-01-27 17:53:33.058] [/Users/balupton/Projects/docpad/out/lib/docpad.js:3562] [performGenerate [as _onTimeout]]
debug: Generate options: { cache: 'write',
  initial: false,
  reset: false,
  populate: false,
  reload: false,
  partial: true,
  renderPasses: 1 }
    → [2014-01-27 17:53:33.061] [/Users/balupton/Projects/docpad/out/lib/docpad.js:3050] [DocPad.generate]
info: Generating...
    → [2014-01-27 17:53:33.064] [/Users/balupton/Projects/docpad/out/lib/docpad.js:3054] [DocPad.generate]
debug: Emitting the event: notify
    → [2014-01-27 17:53:33.067] [/Users/balupton/Projects/docpad/out/lib/docpad.js:1175] [DocPad.emitSerial]
debug: Emitted the event: notify
    → [2014-01-27 17:53:33.073] [/Users/balupton/Projects/docpad/out/lib/docpad.js:1180] [TaskGroup.<anonymous>]
Changed query: { '$or': 
   { mtime: { '$gte': Mon Jan 27 2014 17:53:15 GMT-0500 (EST) },
     '$and': { wtime: null, write: true } } }
mtimes [ Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:30 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:16 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:17 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:17 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:17 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:17 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:17 GMT-0500 (EST),
  Mon Jan 27 2014 17:53:17 GMT-0500 (EST) ]
debug: Emitting the event: generateBefore

@stongo
Copy link

stongo commented Jan 27, 2014

That makes sense actually. Also, just double checked standalone
functionality, and I it's actually 6.59.2 where it breaks.

On 2014-01-27 5:55 PM, Benjamin Arthur Lupton wrote:

It seems it's something busted with the mtimes. They are always
greater than the last generate time...

—
Reply to this email directly or view it on GitHub
#785 (comment).

@balupton
Copy link
Member Author

Problem is here:

            .on('add change:outPath', (model) =>
                # Skip if we are not a writeable file
                return  if model.get('write') is false

                # Check if we have changed our outPath
                previousOutPath = model.previous('outPath')
                if previousOutPath
                    # Ensure we regenerate anything (on the next regeneration) that was using the same outPath
                    previousModels = @database.findAll(outPath:previousOutPath)
                    previousModels.each (model) ->
                        model.set('mtime': new Date())
                    docpad.log('debug', 'Updated mtime for these models due to addition of a similar one', previousModels.pluck('relativePath'))

Need to figure out why this was added. Whether or not it should run for every single document, or whether or not that is a mis-intention.

@balupton
Copy link
Member Author

Fixed in DocPad v6.61.0. Thanks everyone for the help on this.

This was referenced Jan 28, 2014
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
1 - confirmed affects core affects watching contributions welcome difficulty days type defect
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@balupton @dimitarkolev @stongo