Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialize random number generator seed. #1503

Merged
merged 1 commit into from
Apr 27, 2016

Conversation

urso
Copy link

@urso urso commented Apr 27, 2016

Initialize random number generator using cryptographic seed read from OS. Fixes
issues with random number generator always returning same sequence, e.g. when
using random endpoint connecting to logstash in failover mode.

// cryptographically strong pseudo random number generator.
//
// On linux Reader might use getrandom(2) or /udev/random. On windows systems
// GryptGenRandom is used.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/GryptGenRandom/CryptGenRandom/?

@andrewkroh
Copy link
Member

LGTM

Initialize random number generator using cryptographic seed read from OS. Fixes
issues with random number generator always returning same sequence, e.g. when
using random endpoint connecting to logstash in failover mode.
@urso urso force-pushed the fix/random-seed-init branch from ddc0252 to c81a91a Compare April 27, 2016 14:46
@codecov-io
Copy link

codecov-io commented Apr 27, 2016

Current coverage is 33.38%

Merging #1503 into master will decrease coverage by -19.60%

@@             master      #1503   diff @@
==========================================
  Files           197         55     -142   
  Lines         22266       4242   -18024   
  Methods           0          0            
  Messages          0          0            
  Branches          0          0            
==========================================
- Hits          11796       1416   -10380   
+ Misses         9517       2716    -6801   
+ Partials        953        110     -843   
  1. 4 files (not in diff) in packetbeat/sniffer were deleted. more
  2. 2 files (not in diff) in ...etbeat/protos/thrift were deleted. more
  3. 2 files (not in diff) in ...ketbeat/protos/redis were deleted. more
  4. 2 files (not in diff) in ...ketbeat/protos/pgsql were deleted. more
  5. 6 files (not in diff) in ...acketbeat/protos/nfs were deleted. more
  6. 3 files (not in diff) in ...tbeat/protos/mongodb were deleted. more
  7. 8 files (not in diff) in ...beat/protos/memcache were deleted. more
  8. 4 files (not in diff) in ...cketbeat/protos/icmp were deleted. more
  9. 2 files (not in diff) in ...cketbeat/protos/http were deleted. more
  10. 5 files (not in diff) in ...acketbeat/protos/dns were deleted. more

Sunburst

Powered by Codecov. Last updated by 47dc64b

@tsg tsg merged commit 5bfba72 into elastic:master Apr 27, 2016
0-wiz-0 referenced this pull request in NetBSD/pkgsrc-wip Nov 30, 2016
Beats version 5.0.2

Metricbeat
- Fix the `password` option in the MongoDB module.

Filebeat
- Fix registry cleanup issue when files falling under ignore_older after
  restart.

Beats version 5.0.1

Metricbeat
- Fix system.process.start_time on Windows.
- Fix system.process.ppid on Windows.
- Fix system process metricset for Windows XP and 2003. cmdline will be
  unavailable.
- Fix access denied issues in system process metricset by enabling
  SeDebugPrivilege on Windows.
- Fix system diskio metricset for Windows XP and 2003.
- Add username and password config options to the PostgreSQL module.
- Add username and password config options to the MongoDB module.
- Add system core metricset for Windows.

Packetbeat
- Fix 'index out of bounds' bug in Packetbeat DNS protocol plugin.
- Define client_geoip.location as geo_point in the mappings to be used
  by the GeoIP processor in the Ingest Node pipeline.

Filebeat
- Stop Filebeat on registrar loading error.

Beats version 5.0.0-GA

Affecting all Beats
- Fix kafka output re-trying batches with too large events.
- Fix kafka output protocol error if version: 0.10 is configured.
- Fix kafka output connection closed by broker on SASL/PLAIN.
- Add beat.version fields to all events.

Metricbeat
- Fix high CPU usage on macOS when encountering processes with long
  command lines.
- Fix high value of system.memory.actual.free and
  system.memory.actual.used.
- Change several OpenProcess calls on Windows to request the lowest
  possible access provilege.
- Fix system.memory.actual.free high value on Windows.

Filebeat
- Fix issue when clean_removed and clean_inactive were used together
  that states were not directly removed from the registry.
- Fix issue where upgrading a 1.x registry file resulted in duplicate
  state entries.

Beats version 5.0.0-rc1

Affecting all Beats
- A dynamic mapping rule is added to the default Elasticsearch template
  to treat strings as keywords by default.
- Make sure Beats sent always float values when they are defined as
  float by sending 5.00000 instead of 5.
- Fix ignoring all fields from drop_fields in case the first field is
  unknown.
- Fix dynamic configuration int/uint to float type conversion.
- Fix primitive types conversion if values are read from environment
  variables.

Metricbeat
- Fix default configuration file on Windows to not enabled the load
  metricset.

Packetbeat
- Fix the bpf_filter setting.

Filebeat
- Fix input buffer on encoding problem.
- Fix registry cleanup issue when files falling under ignore_older after
  restart.

Beats version 5.0.0-beta1

Affecting all Beats
- Change Elasticsearch output index configuration to be based on format
  strings. If index has been configured, no date will be appended
  anymore to the index name.
- Replace output.kafka.use_type by output.kafka.topic accepting a format
  string.
- If the path specified by the -c flag is not absolute and -path.config
  is not specified, it is considered relative to the current working
  directory.
- rename tls configurations section to ssl.
- rename certificate_key configuration to key.
- replace tls.insecure with ssl.verification_mode setting.
- replace tls.min/max_version with ssl.supported_protocols setting
  requiring full protocol name.
- Fix logstash output handles error twice when asynchronous sending
  fails.
- Fix Elasticsearch structured error response parsing error.
- Fixed the run script to allow the overriding of the configuration
  file.
- Fix logstash output crash if no hosts are configured.
- Fix array value support in -E CLI flag.
- Fix merging array values if -c CLI flag is used multiple times.
- Fix beats failing to start due to invalid duplicate key error in
  configuration file.
- Fix panic on non writable logging directory.
- Add script to generate the Kibana index-pattern from fields.yml.
- Enhance Redis output key selection based on format string.
- Configurable Redis keys using filters and format strings.
- Add format string support to output.kafka.topic.
- Add output.kafka.topics for more advanced kafka topic selection per
  event.
- Add support for Kafka 0.10.
- Add SASL/PLAIN authentication support to kafka output.
- Make Kafka metadata update configurable.
- Add Kafka version setting (optional) enabling kafka broker version
  support.
- Add Kafka message timestamp if at least version 0.10 is configured.
- Add configurable Kafka event key setting.
- Add settings for configuring the kafka partitioning strategy.
- Add partitioner settings reachable_only to ignore partitions not
  reachable by network.
- Enhance contains condition to work on fields that are arrays of
  strings.
- Lookup the configuration file relative to the -path.config CLI flag.
- Re-write import_dashboards.sh in Golang.
- Update to Go 1.7.
- Log total non-zero internal metrics on shutdown.
- Add support for encrypted private key files by introducing
  ssl.key_passphrase setting.
- Add experimental symlink support with symlinks config
- Topology map is deprecated. This applies to the settings:
  refresh_topology_freq, topology_expire, save_topology, host_topology,
  password_topology, db_topology.

Metricbeat
- Change field type system.process.cpu.start_time from keyword to date.
- redis/info metricset fields were renamed up according to the naming
  conventions.
- Fix module filters to work properly with drop_event filter.
- Use the new scaled_float Elasticsearch type for the percentage values.
- Add experimental cgroup metrics to the system/process MetricSet.
- Added a PostgreSQL module.
- Improve mapping by converting half_float to scaled_float and integers
  to long.
- Add experimental haproxy module.

Packetbeat
- Group HTTP fields under http.request and http.response
- Export http.request.body and http.response.body when configured under
  include_body_for
- Move ignore_outgoing config to packetbeat.ignore_outgoing
- Fix mapping for some Packetbeat flow metrics that were not marked as
  being longs.
- Fix handling of messages larger than the maximum message size (10MB).
- Add Cassandra protocol analyzer to Packetbeat.
- Match connections with IPv6 addresses to processes
- Add IP address to -devices command output
- Add configuration option for the maximum message size. Used to be
  hard-coded to 10 MB.

Filebeat
- Set close_inactive default to 5 minutes (was 1 hour before)
- Set clean_removed and close_removed to true by default
- Fix processor failure in Filebeat when using regex, contain, or equals
  with the message field.
- Fix async publisher sending empty events
- Fix potential issue with multiple harvester per file on large file
  numbers or slow output
- Introduce close_timeout harvester options
- Strip BOM from first message in case of BOM files
- Add harvester_limit option

Beats version 5.0.0-alpha5

Affecting all Beats
- Rename the filters section to processors.
- Introduce the condition with when in the processor configuration.
- The Elasticsearch template is now loaded by default.
- The Redis output index setting is renamed to key. index still works
  but it's deprecated.
- The undocumented file output index setting was removed. Use filename
  instead.
- Fix sync publisher PublishEvents return value if client is closed
  concurrently.
- Periodically log internal metrics.
- Add enabled setting to all output modules.
- Command line flag -c can be used multiple times.
- Add OR/AND/NOT to the condition associated with the processors.
- Add -E CLI flag for overwriting single config options via command
  line.
- Choose the mapping template file based on the Elasticsearch version.
- Check stdout being available when console output is configured.

Metricbeat
- Create a separate metricSet for load under the system module and
  remove load information from CPU stats.
- Add system.load.norm.1, system.load.norm.5 and system.load.norm.15.
- Add threads fields to mysql module.
- Do not send zero values when no value was present in the source.
- Add pgid field to process information.
  https://github.com/elastic/beats/pull/ 2021[2021]

Packetbeat
- Set enabled ` in packetbeat.protocols.icmp configuration to true by
  default.
- Add enabled setting to Packetbeat protocols.
- Add enabled setting to Packetbeat network flows configuration.

Filebeat
- Fix potential data loss between Filebeat restarts, reporting
  unpublished lines as published.
- Fix open file handler issue. 2028
- Fix filtering of JSON events when using integers in conditions.
- Introduce close_removed and close_renamed harvester options.
- Introduce close_eof harvester option.
- Add clean_removed and clean_inactive config option.
- Deprecate close_older option and replace it with close_inactive.
- Deprecate force_close_files option and replace it with close_removed
  and close_renamed.

Beats version 5.0.0-alpha4

Affecting all Beats
- The topology_expire option of the Elasticserach output was removed.
- Reset backoff factor on partial ACK.
- Fix beats load balancer deadlock if max_retries: -1 or publish_async
  is enabled in filebeat.
- Fix logstash output with pipelining mode enabled not reconnecting.
- Empty configuration sections become merge-able with variables
  containing full path.
- Fix error message about required fields missing not printing the
  missing field name.
- Improve error message if compiling regular expression from config
  files fails.
- Compression support in the Elasticsearch output.

Metricbeat
- Fix the CPU values returned for each core.
- Add MongoDB module.

Packetbeat
- Add missing nil-check to memcached GapInStream handler.
- Fix NFSv4 Operation returning the first found first-class operation
  available in compound requests.
- Fix TCP overlapping segments not being handled correctly.

Filebeat
- Stop following symlink. Symlinks are now ignored:

Beats version 5.0.0-alpha3

Affecting all Beats
- All configuration settings under shipper: are moved to be top level
  configuration settings. I.e. shipper.name: becomes name: in the
  configuration file.
- Add conditions to generic filtering.
- The support for doing GeoIP lookups is deprecated and will be removed
  in version 6.0.

Topbeat
- Topbeat is replaced by Metricbeat.

Metricbeat
- First public release, containing the following modules: apache, mysql,
  nginx, redis, system, and zookeeper.

Filebeat
- The state for files which fall under ignore_older is not stored
  anymore. This has the consequence, that if a file which fell under
  ignore_older is updated, the whole file will be crawled.
- The registry format was changed to an array instead of dict. The
  migration to the new format will happen automatically at the first
  startup.

Beats version 5.0.0-alpha2

Affecting all Beats
- On DEB/RPM installations, the binary files are now found under
  /usr/share/{{beat_name}}/bin, not in /usr/bin.
- The logs are written by default to self rotating files, instead of
  syslog.
- Remove deprecated host option from elasticsearch, logstash and redis
  outputs.
- Drain response buffers when pipelining is used by Redis output.
- Unterminated environment variable expressions in config files will now
  cause an error
- Fix issue with the automatic template loading when Elasticsearch is
  not available on Beat start.
- Fix bug affecting -cpuprofile, -memprofile, and -httpprof CLI flags
- Fix race when multiple outputs access the same event with logstash
  output manipulating event 1410
- Seed random number generator using crypto.rand package.
  https://github.com/elastic/beats/pull/1503{1503]
- Fix beats hanging in -configtest
- Fix kafka log message output
- Add support for TLS to Redis output.
- Add SOCKS5 proxy support to Redis output.
- Failover and load balancing support in redis output.
- Multiple-worker per host support for redis output.
- Added ability to escape ${x} in config files to avoid environment
  variable expansion
- Configuration options and CLI flags for setting the home, data and
  config paths.
- Configuration options and CLI flags for setting the default logs path.
- Update to Go 1.6.2
- Add Elasticsearch template files compatible with Elasticsearch 2.x.
- Add scripts for managing the dashboards of a single Beat

Packetbeat
- Configuration of redis topology support changed.
- Move all Packetbeat configuration options under the packetbeat
  namespace
- Fix compile issues for OpenBSD.

Filebeat
- Default location for the registry file was changed to be data/registry
  from the binary directory, rather than .filebeat in the current
  working directory. This affects installations for zip/tar.gz/source, the
  location for DEB and RPM packages stays the same.
- Improvements in registrar dealing with file rotation.
- Fix issue with JSON decoding where @timestamp or type keys with the
  wrong type could cause Filebeat to crash.
- Fix issue with JSON decoding where values having null as values could
  crash Filebeat.
- Multiline reader normalizing newline to use \n.

Topbeat
- Updated elastic/gosigar version so Topbeat can compile on OpenBSD.

Beats version 5.0.0-alpha1

libbeat
- Run function to start a Beat now returns an error instead of directly
  exiting.
- The method signature of HandleFlags() was changed to allow returning
  an error
- Require braces for environment variable expansion in config files

Packetbeat
- Rename output fields in the dns package. Former flag recursion_allowed
  becomes recursion_available. 803 Former SOA field ttl becomes minimum.
- The fully qualified domain names which are part of output fields
  values of the dns package now terminate with a dot.
- Remove the count field from the exported event

Topbeat
- Rename proc.cpu.user_p with proc.cpu.total_p as it includes CPU time
  spent in kernel space
- Remove count field from the exported fields
- Rename input top level config option to topbeat

Filebeat
- Scalar values in used in the fields configuration setting are no
  longer automatically converted to strings.
- Count field was removed from event as not used in filebeat

Beats version 1.3.1

Filebeat
- Fix a concurrent bug on filebeat startup with a large number of
  prospectors defined.

Packetbeat
- Fix description for the -I CLI flag.

Beats version 1.3.0

Affecting all Beats
- Fix beats load balancer deadlock if max_retries: -1 or publish_async
  is enabled in filebeat.
- Fix output modes backoff counter reset. 1803 1814
- Set logstash output default bulk_max_size to 2048.
- Seed random number generator using crypto.rand package.
- Check stdout being available when console output is configured.
- Updated to Go 1.7

Packetbeat
- Add missing nil-check to memcached GapInStream handler.
- Fix NFSv4 Operation returning the first found first-class operation
  available in compound requests.
- Fix TCP overlapping segments not being handled correctly.

Filebeat
- Undocumented support for following symlinks is deprecated. Filebeat
  will not follow symlinks in version 5.0.

Beats version 1.2.3

Topbeat
- Fix high CPU usage when using filtering under Windows.

Filebeat
- Fix rotation issue with ignore_older.
- Prevent file opening for files which reached ignore_older.
@urso urso deleted the fix/random-seed-init branch February 19, 2019 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants