This repository has been archived by the owner on May 8, 2024. It is now read-only.
API change
- Remove module-level api like
dpark.textFile
. - Support Streaming shuffle and Disk shuffle (Experimental, compatible).
Fixes
- Bug when parsing mfs chunk info.
Improvement
- Better broadcast impl using shared memory for tasks on the same slave to reduce memory cost.
- Better offer-matching logic for MesosScheduler which remember bad slaves.
- Refactor: style and layout.
New Feature
- Multi segment dump to save memory.
- Gather statics for stage.
- Support run tests/test_rdd on mesos.
- Add colorful progress bar for dpark.
- Support mesos role.
- Support multi named mesos master in conf.
- Loghub for admin.