sc2reader is a python library for extracting information from various different Starcraft II resources. These resources currently include Replays, Maps, and Game Summaries; we may eventually include BNet profiles and possibly adapters to the more entrenched SCII sites like sc2ranks.
Our goal is to give anyone and everyone the power to construct their own tools, do their own analysis, and hack on their own Starcraft II projects under the open MIT license. Currently powering:
- Websites: ggtracker.com, gamereplays.org
- Tools: The Core
- Research: Build Order Classification
Our secondary goal is to become a reference implementation for people looking to implement parsers in other languages. The following is a list of partial implementations in other languages:
- C#: sc2replay-csharp (special thanks for v1.5 help)
- C++: sc2pp
- Javascript: comsat
- PHP: phpsc2replay
Unfortunately sc2reader does not implement a battle.net scraper for profile information. If you need the information I know of two projects that can get you started:
- Ruby: bnet_scraper - Maintained by Agora Games
- Python: sc2profile - Currently unmaintained, slightly dated.
If you'd like your tool, site, project, or implementation listed above, drop us a line on our mailing list or stop by our #sc2reader IRC channel and say hi!
sc2reader is currently capable of parsing varying levels of information out of the three primary resource types listed below. For a more detailed and exact description of the information that can be extracted please consult the documentation hosted on Read the Docs.
Replays can be parsed for the following general types of information:
- Replay details (map, length, version, expansion, datetime, game type/speed, ...)
- Player details (name, race, team, color, bnet url, win/loss, ...)
- Message details (text, time, player, target, pings, ...)
Additional information can be parsed from ladder replays and replays from basic unmodded private "custom" games. This information includes:
- Unit Selection and Hotkey (Control Group) events.
- Resource Transfers and Requests (but not collection rate or unspent totals!)
- Unfiltered Unit commands (attack, move, train, build, psi storm, etc)
- Camera Movements for all players and observers.
In general, all player actions are recorded but game state information is not. In some cases, further game state information can be extracted from this raw information:
- All unit selections and hotkey values for every frame of the game.
- APM/EPM and its untold variations.
We are in the process of building data dictionaries for all the SC2 units and abilities which should enable much more creative and robust analysis of the raw event stream found in replay files.
Maps are currently parsed in the most minimal way possible and are an area for big improvement in the future. Currently the Minimap.tga packaged with the map is made available along with Name, Author, Description for ONLY enUS localized SCII map files. More robust Map meta data extraction will likely be added in future releases.
There's a lot more in here to be had for the adventurous.
Tons of data parsed. Thank you Prillan and others from Team Liquid.
- Lobby Properties (game speed, game type, ...)
- Player Information (Race, Team, Result, bnet info, ...)
- Player Graphs & Stats (Army Graph, Income Graph, Avg Unspent Resources, ...)
- URLs to map localization files and images
- Player build orders up to 64 (real) actions
Parsing on these files is now production ready for those that can get them. See the Team Liquid thread for details on how to go about getting them.
To demonstrate how you might use sc2reader in practice I've included some short contrived scripts below. For more elaborate examples, checkout the docs and the sc2reader.scripts package on Github or in the source.
Save all the minimaps for all the games you've ever played:
import sc2reader, os, sys
if not os.path.exists('minimaps'):
os.makedirs('minimaps')
# Only load details file (level 1) and fetch the map file from bnet
for replay in sc2reader.load_replays(sys.argv[1:], load_map=True, load_level=1):
minimap_path = os.path.join('minimaps', replay.map_name.replace(' ','_')+'.tga')
if not os.path.exists(minimap_path):
with open(minimap_path, 'w') as file_out:
file_out.write(replay.map.minimap)
print("Saved Map: {0} [{1}]".format(replay.map_name, replay.map_hash))
Organizing your 1v1 replays by race played and matchup and sortable by length:
import sc2reader, os, shutil, sys
sorted_base = 'sorted'
path_to_replays = 'my/replays'
for replay in sc2reader.load_replays(sys.argv[1], load_level=1):
if replay.real_type != '1v1':
continue
try:
me = replay.player.name('ShadesofGray')
you = team[(me.team.number+1)%2].players[0]
matchup = "{0}v{1}".format(me.play_race[0], you.play_race[1])
sorted_path = os.path.join(sorted_base,me.play_race[0],matchup)
if not os.path.exists(sorted_path):
os.makedirs(sorted_path)
filename = "{0} - {1}".format(replay.game_length, replay.filename)
src = os.join(path_to_replays,replay.filename)
dst = os.join(sorted_path, filename)
shutil.copyfile(src, dst)
except KeyError as e:
continue # A game I didn't play in!
Install from the latest release on PyPI with pip:
pip install sc2reader
or easy_install:
easy_install sc2reader
or with setuptools (specify a valid x.x.x):
wget http://pypi.python.org/packages/source/s/sc2reader/sc2reader-x.x.x.tar.gz tar -xzf sc2reader-x.x.x.tar.gz cd sc2reader-x.x.x python setup.py install
Releases to PyPi can be very delayed, for the latest and greatest you are encouraged to install from Github master which is usually kept quite stable.
Github master is generally stable with development branches more unstable.
Install from the latest source on github with pip:
pip install -e git+git://github.com/GraylinKim/sc2reader#egg=sc2reader
or with setuptools:
wget -O sc2reader-master.tar.gz https://github.com/GraylinKim/sc2reader/tarball/master tar -xzf sc2reader-master.tar.gz cd sc2reader-master python setup.py install
Contributors should install from an active git repository using setuptools in develop mode. This will install links to the live code so that local edits are available to external modules automatically:
git clone https://github.com/GraylinKim/sc2reader.git cd sc2reader python setup.py develop
Please review the CONTRIBUTING.md file and get in touch with us before doing too much work. It'll make everyone happier in the long run.
We use py.test for testing. You can install it via pip/easy_install:
pip install pytest easy_install pytest
To run the tests just use:
py.test # Runs all the tests py.test test_replays # Only run tests on replays py.test test_s2gs # Only run tests on summary files
When repeatedly running tests it can be very helpful to make sure you've set a local cache directory to prevent long fetch times from battle.net:
mkdir local_cache export SC2READER_CACHE_DIR=local_cache
Good luck, have fun!
sc2reader has a small but growing community of people looking to make tools and websites with Starcraft II data. If that sounds like something you'd like to be a part of please join our underused mailing list and start a conversation or stop by #sc2reader on FreeNode and say 'Hi'. We have members from all over Europe, Australia, and the United States currently, so regardless of the time, you can probably find someone to talk to.
We have an issue tracker on Github that all bug reports and feature requests should be directed to. We have a mailing list with Google Groups that you can use to reach out for support. We are generally on FreeNode in the #sc2reader and can generally provide live support and address issues there as well.
Thanks to all the awesome developers in the SC2 community that helped out and kept this project going. Special thanks to the people of the awesome phpsc2replay project whose public documentation and source code made starting this library possible and to sc2replay-csharp for setting us straight on game events parsing and assisting with our v1.5 upgrade. I'd also like to thank ggtracker sponsoring further development and for providing the thousands of test files used while adding s2gs and HotS Beta support.