I'm so sorry that I didn't update anything these days. Because of moving and starting new job I'm so busy. The next update or better documentation will come whenever I would really have time.
This is a project for my master's thesis. Anyone who is interested in making powerful Mahjong AI is welcome to extend my agent. For more details about the applied algorithms and the reasons why such algorithms are used, please contact me with E-Mail.
In the case that you want to develop your own Mahjong agent, this Repo can also be used as a Framework for realtime testing (with real human players). Then you can save all your time for finding the best strategy for your agent. Furthermore a development library (crawling and pre-processing of game logs, calculation of shantin and winning score etc.) is now available in: https://github.com/erreurt/MahjongKit
Next Update coming soon:
-
Better feature engineering needed Decide whether to call a meld/call Riichi or not using Random Forest (Instead of condition-rules).
-
Ongoing: Training waiting tiles prediction model with LSTM.
Attention: If you test your bot parallelly with more than 4 accounts under the same IP adresse, your IP adress will be banned by tenhou.net for 24 hours. (I don't know exactly about the rules of banning players, but that's all inferred from my observation.)
| Author | Jianyang Tang (Thomas) |
|---|---|
| jian4yang2.tang1@gmail.com |
Mahjong is a four player strategy game with imperfect information. The main challenges of developing an intelligent Mahjong agent are for example the complicated game rules, the immense search space, multiple opponents and imperfect information. Several existing works have attempted to tackle these problems through Monte Carlo tree simulation, utility function fitting by supervised learning or opponents model by regression algorithms. Nevertheless, the performance of intelligent Mahjong playing agents is still far away from that of the best human players. Based on statistical analysis for the Mahjong game and human expert knowledge, an intelligent Mahjong agent was proposed in this work. To tackle the problems of the state-of-the-art work, heuristics technologies and an enhanced opponents model achieved by adopting bagging of multilayer perceptrons were applied to this work. The experiments show that the proposed agent outperforms the state-of-the-art agent, and the applied opponents model has a significant positive effect on the performance of the agent. Furthermore, several interesting points can be discerned from the experiments, which are pretty meaningful for future work.
- 1. Japanese Riichi Mahjong: Game rules
- 2. Directory of the program
- 3. Overview of the Mahjong bot
- 4. Experiments result of the bot
- 5. Instructions I: Run the proposed Mahjong bot
- 6. Instructions II: Develop your own Mahjong bot
- Acknowledgments
Refer to https://en.wikipedia.org/wiki/Japanese_Mahjong for game rules of Japanese Riichi Mahjong.
The implemented client allows one to run a Mahjong agent directly through the programm, instead of doing this in the web browser. The site for online playing of Japanese Riichi Mahjong is http://tenhou.net/
The left side is a typical scenary of Japanese Riichi Mahjong table. This picture is screenshot from the GUI implemented for debugging use.
- [MahjongAI]
- [agents] Agent
- [utils]
- [clfs]
- multiple multilayer perceptrons classifiers several MLP classifiers for predicting dangerous tiles
- ......
- wait_calc.py contains functions that calculate whether the hand tiles are in waiting state, return the corresponding winning score, the partition, the to be discarded tile
- win_calc.py contains functions that calculate the winning score for a specific partition of hand tiles
- [clfs]
- ai_interface.py the class that should be inherited while developing your own Mahjong agent
- experiment_ai.py the old agent which was used in the experiments
- jianyang_ai.py the up-to-date agent
- random_ai_example.py the example Mahjong agent class for giving a simple explanation on how to develop your own Mahjong agent
- [utils]
- [client] For realtime testing with human beings
- [tilespng] Pics used for GUI
- mahjong_meld.py class of Mahjong meld
- mahjong_tile.py class of Mahjong tile
- mahjong_table.py class of Mahjong game table, for storing game state
- mahjong_player.py class of Mahjong player, for storing players' tiles and game status
- tenhou_client.py class of client for tenhou.net. This class implements the game just as the browser version
- tenhou_parser.py class of parser for tenhou.net server. It provides decoding methods for messages reveived from the tenhou.net server
- [logger] Recording the playing history of the agent while testing
- logger_handler.py class of logging, for storing game run and game results
- main.py contains example of how to run the Mahjong agent
- gui_main.py contains example of how to run the Mahjong agent with GUI
- [agents] Agent
The proposed Mahjong agent was tested on tenhou.net. The test was carried out in two versions, i.e. one with defence model and another one without. The raw game logs and intermediate game results can be found in my another repository: https://github.com/erreurt/Experiments-result-of-mahjong-bot. The experiments were carried out with the agent version in experiment_ai.py.
For the version with defence model, 526 games were played, and for the version without defence model, 532 games were played. This is not as many as two related works, but as shown in the figure of the convergence behavior of agent's performance, 526 games are sufficient.
Mizukami's extended work can be seen as currently the best and the most reliable Mahjong agent in English literatures. Here a comparison between the performance of my Mahjong agent and that of Mizukami's is presented:
| [1] | [2] | [3] | [4] | |
|---|---|---|---|---|
| Games played | 526 | 532 | 2634 | 1441 |
| 1st place rate | 23.95% | 22.65% | 24.10% | 25.30% |
| 2nd place rate | 26.62% | 25.92% | 28.10% | 24.80% |
| 3rd place rate | 31.75% | 25.71% | 24.80% | 25.10% |
| 4th place rate | 17.68% | 25.71% | 23.00% | 24.80% |
| win rate | 24.68% | 26.50% | 24.50% | 25.60% |
| lose rate | 13.92% | 20.21% | 13.10% | 14.80% |
| fixed level | 2.21 Dan | 0.77 Dan | 1.14 Dan | 1.04 Dan |
-
[1] My Mahjong agent with defence model
-
[2] My Mahjong agent without defence model
-
[3] Mizukami's extended work: Mizukami N., Tsuruoka Y.. Building a computer mahjong player based on monte carlo simulation and opponent models. In: 2015 IEEE Conference on Computational Intelligence and Games (CIG), pp. 275–283. IEEE (2015)
-
[4] Mizukami et. al.: N. Mizukami, R. Nakahari, A. Ura, M. Miwa, Y. Tsuruoka, and T. Chikayama. Realizing a four-player computer mahjong program by supervised learning with isolated multi-player aspects. Transactions of Information Processing Society of Japan, vol. 55, no. 11, pp. 1–11, 2014, (in Japanese).
Note that while playing Mahjong, falling into 4th place is definitely a taboo, since one would get level points reduced. As a result, the rate of falling into 4th place of a player is critical to its overall performance. My bot has a better fixed level exactly due to the low 4th place rate.
To run the Mahjong agent, one has to specify a few configurations. As shown in the following example from main.py:
def run_example_ai():
ai_module = importlib.import_module("agents.random_ai_example")
ai_class = getattr(ai_module, "RandomAI")
ai_obj = ai_class() # [1]
player_module = importlib.import_module("client.mahjong_player")
opponent_class = getattr(player_module, "OpponentPlayer") # [2]
user = "ID696E3BCC-hLHNE8Wf" # [3]
user_name = "tst_tio" # [4]
game_type = '1' # [5]
logger_obj = Logger("log1", user_name) # [6]
connect_and_play(ai_obj, opponent_class, user, user_name, '0', game_type, logger_obj) # play one gamedef run_jianyang_ai():
ai_module = importlib.import_module("agents.jianyang_ai")
waiting_prediction_class = getattr(ai_module, "EnsembleCLF")
ensemble_clfs = waiting_prediction_class()
ai_class = getattr(ai_module, "MLAI")
ai_obj = ai_class(ensemble_clfs) # [1]
opponent_class = getattr(ai_module, "OppPlayer") # [2]
user = "ID696E3BCC-hLHNE8Wf" # [3]
user_name = "tst_tio" # [4]
game_type = '1' # [5]
logger_obj = Logger("log_jianyang_ai_1", user_name) # [6]
connect_and_play(ai_obj, opponent_class, user, user_name, '0', game_type, logger_obj)-
AI instance: A class instance of the Mahjong agent. In this repository three versions of Mahjong agent are provided. The first one is in agents.random_ai_example.py, this is a demo class for showing potential developers how to implement his/her own agents. The second one is in agents.experiment_ai.py and the experiment results given in part 4 is generated by this AI. The third one is the up-to-date AI and is in agents.jianyang_ai.py.
-
Opponent player class: The class of Opponent player. One can use the default class OpponentPlayer in client.mahjong_player. If one has extended the OpponentPlayer class due to extra needs, this variable should be set to your corresponding class.
-
User ID: A token in the form as shown in the example that one got after registration on tenhou.net. ATTENTION: Please use your own user ID. If the same ID is used under different IP address too often, the account will be temperorily blocked by tenhou.net.
-
User name: The corresponding user name you have created while registrating on tenhou.net. This variable is only for identifying your test logs.
-
Game type: The game type is encoded as a 8-bit integer. Followings are the description for each bit.
- 0-th: 1 play with online players, 0 play with robots
- 1-th: 1 no red bonus tiles, 0 with red bonus tiles
- 2-th: 1 no open tanyao, 0 with open tanyao
- 3-th: 1 only east+south round (regurarily 8 rounds), 0 only east round (4 rounds)
- 4-th: 1 three players (two opponents), 0 four players (three opponents)
- 6-th: 1 fast game (7s for decision making), 0 slow game (15s)
- 5-th & 7-th: game room 00-starter, 01-upper, 10-mega upper, 11-phoenix
For examples:
- "1" = "00000001": with online real human players, with red bonus tiles, with open tanyao, 4 rounds, 4 players, slow game, starter game room
- "137" = "10001001": with online real human players, with red bonus tiles, with open tanyao, 8 rounds, 4 players, slow game, upper game room
- "193" = "11000001": with online real human players, with red bonus tiles, with open tanyao, 4 rounds, 4 players, fast game, upper game room
- Tenhou.net does not provide all possibility of the above specified combinations. Most online players play on configurations for example "1", "137", "193", "9" -
Logger: Two parameters are required for initialising the logger. The first one is the user-defined logger's ID, such that developers can freely name his/her test history.
After specifying all these configurations, just throw all these parameters to connect_and_play(). Then it's time watch the show of your Mahjong agent!!!
Four functions must be implemented for the Mahjong bot, as shown in the "interface" class in agents.ai_interface. It is recommended that your agent is an inheritance of the AIInterface. For a deeper explanation and a simple example of these functions, please see documentation in agents.random_ai_example.py.
class AIInterface(MainPlayer):
def to_discard_tile(self):
raise NotImplementedError
def should_call_kan(self, tile136, from_opponent):
raise NotImplementedError
def try_to_call_meld(self, tile136, might_call_chi):
raise NotImplementedError
def can_call_reach(self):
raise NotImplementedError-
to_discard_tile: Based on all the accessible information about the game state, this function returns a tile to discard. The return is an integer in the range 0-135. There are toally 136 tiles in the Mahjong game, i.e. 34 kinds of tiles and 4 copies for each kind. In different occasions we use either the 34-form (each number corresponds to one kind of tile) or the 136-form (each number corresponds to a tile). Note that here the return should be in the 136-form.
-
should_call_kan: https://en.wikipedia.org/wiki/Japanese_Mahjong#Making_melds_by_calling. This function should decide whether the agent should call a kan(Quad) meld. tile136 stands for the tile that some opponent has discarded, which can be used for the agent to form the kan meld. from_opponent indicates whether the agent forms the kan meld by opponent's discard (three tiles in hand and the opponent discards the fourth one) or own tiles(all four tiles in hand).
-
try_to_call_meld: https://en.wikipedia.org/wiki/Japanese_Mahjong#Making_melds_by_calling. This function decides whether the agent should call a Pon(Triplet)/Chi(Sequence) meld. tile136 stands for the tile in 136-form that some opponents has discarded. might_call_chi indicates whether the agent could call a Chi meld, since a Chi meld can only be called with discard of opponent in the left seat.
-
can_call_reach: https://en.wikipedia.org/wiki/Japanese_Mahjong#R%C4%ABchi. This function decides whether the agent should claim Riichi.
When the Mahjong agent class is a subclass of the AIInterface class, information listed as follows can be accessed inside the agent class as specified.
| Access | Data type | Mutable | Desription |
|---|---|---|---|
| self.tiles136 | list of integers | Y | hand tiles in 136-form |
| self.hand34 | list of integers | N | hand tiles in 34-form (tile34 = tile136//4) |
| self.discard136 | list of integers | Y | the discards of the agent in 136-from |
| self.discard34 | list of integers | N | the discards of the agent in 34-form |
| self.meld136 | list of Meld instances | Y | the called melds of the agent, instances of class Meld in client.mahjong_meld.py |
| self.total_melds34 | list of list of integers | N | the called melds of the agent in 34-form |
| self.meld34 | list of list of integers | N | the called pon/chow melds of the agent in 34-form |
| self.pon34 | list of list of integers | N | the called pon melds of the agent in 34-form |
| self.chow34 | list of list of integers | N | the called chow melds of the agent in 34-form |
| self.minkan34 | list of list of integers | N | the called minkan melds of the agent in 34-form |
| self.ankan34 | list of list of integers | N | the called ankan melds of the agent in 34-form |
| self.name | string | Y | name of the account |
| self.level | string | Y | level of the account |
| self.seat | integer | Y | seat ID, the agent always has 0 |
| self.dealer_seat | integer | Y | the seat ID of the dealer |
| self.is_dealer | boolean | N | whether the agent is dealer or not |
| self.reach_status | boolean | Y | indicates whether the agent has claimed Riichi |
| self.just_reach() | boolean | N | whether the agent just claimed Riichi |
| self.tmp_rank | integer | Y | rank of the agent in the current game |
| self.score | integer | Y | score of the agent in the current game |
| self.is_open_hand | boolean | N | whether the agent has already called open melds |
| self.turn_num | integer | N | the number of the current turn |
| self.player_wind | integer | N | player wind is one kind of yaku |
| self.round_wind | integer | N | round wind is one kind of yaku |
| self.bonus_honors | list of integers | Y | all the character tiles which have yaku |
One can access to the instance of opponent class by calling self.game_table.get_player(i) with i equals 1,2,3, which indicates the corresponding id of the opponent.
| Access | Data type | Mutable | Desription |
|---|---|---|---|
| .discard136 | list of integers | Y | the discards of the observed opponent in 136-from |
| .discard34 | list of integers | N | the discards of the observed opponent in 34-form |
| .meld136 | list of Meld instances | Y | the called melds of the observed opponent |
| .total_melds34 | list of list of integers | N | the called melds of the observed opponent in 34-form |
| .meld34 | list of list of integers | N | the called pon/chow melds of the observed opponent in 34-form |
| .pon34 | list of list of integers | N | the called pon melds of the observed opponent in 34-form |
| .chow34 | list of list of integers | N | the called chow melds of the observed opponent in 34-form |
| .minkan34 | list of list of integers | N | the called minkan melds of the observed opponent in 34-form |
| .ankan34 | list of list of integers | N | the called ankan melds of the observed opponent in 34-form |
| .safe_tiles | list of integers | Y | tiles in 34-form which are absolutely safe for the agent, i.e. the observed opponent cannot win with these tiles |
| .name | string | Y | name of the opponent |
| .level | string | Y | level of the opponent |
| .seat | integer | Y | seat ID of the observed opponent |
| .dealer_seat | integer | Y | the seat ID of the dealer |
| .is_dealer | boolean | N | whether the observed opponent is dealer or not |
| .reach_status | boolean | Y | indicates whether the observed opponent has claimed Riichi |
| .just_reach() | boolean | N | whether the observed opponent just claimed Riichi |
| .tmp_rank | integer | Y | rank of the observed opponent in the current game |
| .score | integer | Y | score of the observed opponent in the current game |
| .is_open_hand | boolean | N | whether the observed opponent has already called open melds |
| .turn_num | integer | N | the number of the current turn |
| .player_wind | integer | N | player wind is one kind of yaku |
| .round_wind | integer | N | round wind is one kind of yaku |
| .bonus_honors | list of integers | Y | all the character tiles which have yaku |
To access information about the game table, one can call self.game_table
| Access | Data type | Mutable | Desription |
|---|---|---|---|
| .bot | instance of agent class | Y | class instance of the agent |
| .get_player(i) | instance of opponent class | Y | class instance of the opponent, i=1,2,3 |
| .dealer_seat | integer | Y | seat ID of the dealer |
| .bonus_indicator | list of integers | Y | bonus indicators in 136-form |
| .round_number | integer | Y | round number |
| .reach_sticks | integer | Y | How many Riichi sticks are on the table. The player who win next will receive all the points of these Riichi sticks |
| .honba_sticks | integer | Y | How many honba sticks are on the table. The player will have extra points according to honba sticks if it wins |
| .count_ramaining_tiles | integer | Y | The number of remaining unreavealed tiles |
| .revealed | list of integers | Y | Each element in the list indicates how many copies of this specific tile has been revealed (discards, open melds, bonus indicators etc) |
| .round_win | integer | N | round wind is one kind of yaku |
| .bonus_tiles | list of integers | N | List of bonus tiles. Each occurance of bonus tiles in hand tiles counts as a yaku |
| .last_discard | integer | N | The very latest discard of opponent, this tile is absolutely safe by rules |
-
my professor Johannes Fürnkranz (TU Darmstadt Knowledge Engineering Group)
-
my supervisor Tobias Joppen (TU Darmstadt Knowledge Engineering Group)