Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix data corruption when creating batches. #11

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Wilfred
Copy link
Contributor

@Wilfred Wilfred commented Mar 14, 2015

Previously, we were appending events to the event collection even when
the columns did not match!

Given the following three events:

{
  "name": "my-series",
  "columns": [
    "cpu.user",
    "time"
  ],
  "points": [
    [
      5.0,
      1426288467
    ]
  ]
}
{
  "name": "my-series",
  "columns": [
    "memory.used",
    "time"
  ],
  "points": [
    [
      1000,
      1426288467
    ]
  ]
}
{
  "name": "my-series",
  "columns": [
    "cpu.user",
    "time"
  ],
  "points": [
    [
      6.0,
      1426288477
    ]
  ]
}

We would end up with the following batch:

[
  {
    "name": "my-series",
    "columns": [
      "cpu.user",
      "time"
    ],
    "points": [
      [
        5.0,
        1426288467
      ],
      [
        6.0,
        1426288467
      ]
    ]
  },
  {
    "name": "my-series",
    "columns": [
      "memory.used",
      "time"
    ],
    "points": [
      [
        1000.0,
        1426288467
      ],
      [
        6.0, # this shouldn't be here!
        1426288467
      ]
    ]
  }
]

Previously, we were appending events to the event collection even when
the columns did not match!

Given the following three events:

```ruby
{
  "name": "my-series",
  "columns": [
    "cpu.user",
    "time"
  ],
  "points": [
    [
      5.0,
      1426288467
    ]
  ]
}
{
  "name": "my-series",
  "columns": [
    "memory.used",
    "time"
  ],
  "points": [
    [
      1000,
      1426288467
    ]
  ]
}
{
  "name": "my-series",
  "columns": [
    "cpu.user",
    "time"
  ],
  "points": [
    [
      6.0,
      1426288477
    ]
  ]
}
```

We would end up with the following batch:

```ruby
[
  {
    "name": "my-series",
    "columns": [
      "cpu.user",
      "time"
    ],
    "points": [
      [
        5.0,
        1426288467
      ],
      [
        6.0,
        1426288467
      ]
    ]
  },
  {
    "name": "my-series",
    "columns": [
      "memory.used",
      "time"
    ],
    "points": [
      [
        1000.0,
        1426288467
      ],
      [
        6.0, # this shouldn't be here!
        1426288467
      ]
    ]
  }
]
@brupm
Copy link

brupm commented May 29, 2015

I am not sure I understand this one. @Wilfred by columns do you mean inside the datapoints?

@Wilfred
Copy link
Contributor Author

Wilfred commented May 30, 2015

Yes, inside the datapoints we're taking values from the wrong event. Does that make sense?

@brupm
Copy link

brupm commented May 30, 2015

Yes it does.

@elasticsearch-release
Copy link

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants