{"id":2293,"date":"2025-06-04T22:21:31","date_gmt":"2025-06-04T13:21:31","guid":{"rendered":"https:\/\/www.dogrow.net\/nnet\/?p=2293"},"modified":"2025-06-11T22:47:57","modified_gmt":"2025-06-11T13:47:57","slug":"blog54%e3%80%90othello-ai%e3%80%91%e3%83%a2%e3%83%b3%e3%83%86%e3%82%ab%e3%83%ab%e3%83%ad%e6%b3%95%e3%81%a7-dqn%e3%81%ae%e6%95%99%e5%b8%ab%e3%83%87%e3%83%bc%e3%82%bf%e3%82%92%e4%bd%9c%e3%82%8b%e3%80%82","status":"publish","type":"post","link":"https:\/\/www.dogrow.net\/nnet\/blog54%e3%80%90othello-ai%e3%80%91%e3%83%a2%e3%83%b3%e3%83%86%e3%82%ab%e3%83%ab%e3%83%ad%e6%b3%95%e3%81%a7-dqn%e3%81%ae%e6%95%99%e5%b8%ab%e3%83%87%e3%83%bc%e3%82%bf%e3%82%92%e4%bd%9c%e3%82%8b%e3%80%82\/","title":{"rendered":"(54)\u3010Othello AI\u3011\u30e2\u30f3\u30c6\u30ab\u30eb\u30ed\u6cd5\u3067 DQN\u306e\u6559\u5e2b\u30c7\u30fc\u30bf\u3092\u4f5c\u308b\u3002"},"content":{"rendered":"

\u30101\u3011\u3084\u308a\u305f\u3044\u3053\u3068<\/h1>\n

\u524d\u56de\u306e\u6295\u7a3f\u3067\u6848\u51fa\u3057\u3057\u305f\u5185\u5bb9\u3067\u3001\u5b9f\u88c5\u3092\u9032\u3081\u305f\u3044\u3002
\n(53)\u3010Othello AI\u3011\u30e2\u30f3\u30c6\u30ab\u30eb\u30ed\u6cd5\u3067 DQN\u306e\u6559\u5e2b\u30c7\u30fc\u30bf\u3092\u4f5c\u308d\u3046\u304b\uff1f<\/a><\/p>\n

DQN\u65b9\u5f0f\u30aa\u30bb\u30edAI\u306e\u8a13\u7df4\u30c7\u30fc\u30bf\u3068\u3057\u3066\u3001
\n\u30e2\u30f3\u30c6\u30ab\u30eb\u30ed\u63a2\u7d22\u65b9\u5f0f\u30aa\u30bb\u30edAI\u306e\u6307\u3057\u624b\u5c65\u6b74\u30c7\u30fc\u30bf\u3092\u4f7f\u3044\u305f\u3044\u3002
\n\u307e\u305a\u306f\u53ce\u96c6\u3059\u308b\uff01<\/span><\/p>\n

\u30102\u3011\u3084\u3063\u3066\u307f\u308b<\/h1>\n

1) \u53ce\u96c6\u3059\u308b\u30c7\u30fc\u30bf<\/h2>\n

MCTS vs MCTS\u306e\u5bfe\u5c40\u30c7\u30fc\u30bf\u3092\u4e00\u624b\u3054\u3068\u306b\u4fdd\u5b58\u3059\u308b\u3002
\n\u4fdd\u5b58\u5bfe\u8c61\u306e\u30c7\u30fc\u30bf\u306f\u4e0b\u8a18\u306e\u901a\u308a\u3002
\n\n
#<\/th>\u30d1\u30e9\u30e1\u30fc\u30bf\u540d<\/th>\u8aac\u660e<\/th><\/tr>
1<\/td>state<\/td>\u30aa\u30bb\u30ed\u76e4\u4e0a\u306e\u77f3\u306e\u914d\u7f6e\u3001[6][6]\u306e\u914d\u5217<\/td><\/tr>
2<\/td>action<\/td>\u6307\u3057\u624b (r,c)<\/td><\/tr>
3<\/td>reward<\/td>(state, action) \u306e\u7d50\u679c\u3001\u6307\u3057\u624b(r,c)\u306b\u3088\u308a\u5f97\u3089\u308c\u305f\u5831\u916c<\/td><\/tr>
4<\/td>next_state<\/td>action\u5b9f\u884c\u5f8c\u306e\u30aa\u30bb\u30ed\u76e4\u4e0a\u306e\u77f3\u306e\u914d\u7f6e\u3001[6][6]\u306e\u914d\u5217<\/td><\/tr>
5<\/td>done<\/td>\u30b2\u30fc\u30e0\u7d42\u4e86 or not \u30d5\u30e9\u30b0<\/td><\/tr>\n<\/table><\/p>\n

\u306a\u305c next_state \u304c\u5fc5\u8981\u306a\u306e\uff1f<\/span>
\n\u306f\u3001\u3053\u3061\u3089\uff08\u2193\uff09\u306e\u904e\u53bb\u8a18\u4e8b\u306b\u8a18\u8f09\u3057\u3066\u3044\u308b\u3002DQN\u306e\u6226\u8853\u3067\u4f7f\u3046\u306e\u3060\u3002
\n(45)\u3010Othello AI\u3011\u5f37\u5316\u5b66\u7fd2\u3067\u30aa\u30bb\u30edAI\u3092\u4f5c\u308b\u3002<\/a>
\n\u203b\u4eca\u56de\u306f\u3001\u81ea\u5206\u81ea\u8eab\u306e\u6b21\u306e\u624b\u756a\u76f4\u524d\u306e state\u3067\u306f\u306a\u304f\u3001after-step\uff08\u81ea\u5206\u306e\u6307\u3057\u624b\u76f4\u5f8c\u306e\u76f8\u624b\u8996\u70b9\uff09\u306e state\u3092\u63a1\u7528\u3057\u305f\u3002<\/span> <\/p>\n

\u5bfe\u6226\u3059\u308b MCTS agent\u306e\u63a2\u7d22\u56de\u6570\u306f\u3001\u4ee5\u4e0b\u306e\u5024\u304b\u3089\u9078\u629e\u3059\u308b\u3002\u5024\u304c\u5927\u304d\u3044\u307b\u3069\u63a2\u7d22\u7bc4\u56f2\u304c\u6df1\u304f\u306a\u308a\u3001\u4e00\u822c\u7684\u306b\u306f\u5f37\u304f\u306a\u308b\u3002
\n100, 200, 300, 400, 500, 600, 700, 800, 900, 1000<\/span>
\n\u203b 100\uff5e1000\u306e\u7bc4\u56f2\u3067 100step\u306e 10\u901a\u308a<\/p>\n

\u5f62\u5f0f\u4ed5\u69d8\u8a00\u8a9e\u3060\u3068\u3053\u3093\u306a\u98a8\u306b\u66f8\u3051\u308b\u3002
\nNumbers == { n : \u2115 | 100 \u2264 n \u2227 n \u2264 1000 \u2227 (n mod 100 = 0) }<\/span><\/p>\n

2) \u30d7\u30ed\u30b0\u30e9\u30e0<\/h2>\n

(1) main_CvC_MCTS.py<\/h3>\n

\u6c4e\u7528\u306e COM vs COM\u5bfe\u6226\u30d7\u30ed\u30b0\u30e9\u30e0\u3068\u3057\u3066\u4f5c\u6210\u6e08\u307f\u306e play_2_agent() \u3092\u5b9f\u884c\u3059\u308b\u3002<\/p>\n

\r\nfrom play import play_2_agent\r\nimport argparse\r\nimport pickle\r\nfrom datetime import datetime\r\n\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\ndef main():\r\n    parser = argparse.ArgumentParser(description="COM vs COM MCTS\u7248")\r\n    parser.add_argument("-n",   type=int,   help="\u5bfe\u6226\u56de\u6570", default=1)\r\n    parser.add_argument("--miA",type=int,   help="MCTS(A)\u30d7\u30ec\u30a4\u30a2\u30a6\u30c8\u6570", default=100 )\r\n    parser.add_argument("--miB",type=int,   help="MCTS(B)\u30d7\u30ec\u30a4\u30a2\u30a6\u30c8\u6570", default=100 )\r\n    parser.add_argument("--rh", type=str,   help="\u6307\u3057\u624b\u5c65\u6b74\u30d5\u30a1\u30a4\u30eb\u540d",  default='')\r\n    args = parser.parse_args()\r\n    #---------------------------------------------------------------------------\r\n    recordHistory = True if args.rh != '' else False\r\n    #---------------------------------------------------------------------------\r\n    cOneGameResults = play_2_agent(\r\n        model_pathA   = 'MCTS',\r\n        model_pathB   = 'MCTS',\r\n        n_games       = args.n,\r\n        iterationsA   = args.miA,\r\n        iterationsB   = args.miB,\r\n        recordHistory = recordHistory\r\n    )\r\n    #---------------------------------------------------------------------------\r\n    if recordHistory is True:                       # \u6307\u3057\u624b\u5c65\u6b74\u306e\u8a18\u9332\u6307\u5b9a\u3042\u308a\uff1f\r\n        saveFiles_per_Player(cOneGameResults, args.rh)\r\n\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\ndef saveFiles_per_Player( cOneGameResults, args_rh ):\r\n    transitions = [[], []]                      # \u7a7a\u30ea\u30b9\u30c8\u3067\u521d\u671f\u5316\r\n    for cOneGameResult in cOneGameResults:\r\n        transitions[0] += cOneGameResult.transitions[0]\r\n        transitions[1] += cOneGameResult.transitions[1]\r\n    # --rh \u3067\u6307\u5b9a\u3055\u308c\u305f\u30d5\u30a1\u30a4\u30eb\u540d\u3067 pickle \u306b\u30b7\u30ea\u30a2\u30e9\u30a4\u30ba\r\n    timestamp = datetime.now().strftime("%Y_%m%d_%H%M%S")\r\n    for i, data in enumerate(transitions, start=1):\r\n        fname = f"{args_rh}_P{i}_{timestamp}.pth"\r\n        with open(fname, 'wb') as f:\r\n            pickle.dump(data, f)\r\n        print(f"\u6307\u3057\u624b\u5c65\u6b74\u3092 {fname} \u306b\u4fdd\u5b58\u3057\u307e\u3057\u305f\u3002")\r\n\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\nif __name__ == "__main__":\r\n    main()\r\n<\/pre>\n
(2) main_Collect_MCTS_training_data.py<\/h3>\n
100\uff5e1000\uff08100step\uff09\u306e MCTS\u63a2\u7d22\u4e0a\u9650\u5024\u306e\u7d44\u5408\u305b\u3067\u3001MCTS-agent(A) vs MCTS-agent(B) \u306e\u5bfe\u6226\u3092\u5b9f\u884c\u3059\u308b\u3002
\n\u30b3\u30de\u30f3\u30c9\u30e9\u30a4\u30f3\u5f15\u6570\uff08\uff1d\u5165\u529b\u30d1\u30e9\u30e1\u30fc\u30bf\uff09\u3092\u5909\u3048\u3066\u3001\u4e0a\u8a18\u306e main_CvC_MCTS.py \u3092 CPU 10\u30b3\u30a2\u4e26\u5217<\/span> \u3067\u7e70\u308a\u8fd4\u3057\u5b9f\u884c\u3059\u308b\u3002
\n\u30e2\u30f3\u30c6\u30ab\u30eb\u30ed\u63a2\u7d22\u3067\u306f GPU\u306b\u3088\u308b\u30c7\u30fc\u30bf\u4e26\u5217\u51e6\u7406\u306e\u6069\u6075\u304c\u53d7\u3051\u3089\u308c\u306a\u3044\u306e\u3067\u3001\u51e6\u7406\u4e26\u5217\u304c\u53ef\u80fd\u4e14\u3064\u5358\u4f53\u3067\u9ad8\u901f\u306a CPU\u5b9f\u884c\u3068\u3059\u308b\u3002 \u2192 \u4eca\u56de\u306f\u30b0\u30e9\u30dc\u3092\u4f7f\u308f\u306a\u3044\u3002<\/span><\/p>\n
\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\n# MCTS vs MCTS\u3067\u5bfe\u6226\u3092\u7e70\u308a\u8fd4\u3057\u3001DQN\u5b66\u7fd2\u3067\u4f7f\u7528\u3059\u308b\u6307\u3057\u624b\u5c65\u6b74\u30c7\u30fc\u30bf\u3092\u53ce\u96c6\u3059\u308b\u3002\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\nimport subprocess                       # \u5916\u90e8\u30d7\u30ed\u30b0\u30e9\u30e0\u3092\u547c\u3073\u51fa\u3059\u305f\u3081\u306e\u30e9\u30a4\u30d6\u30e9\u30ea\r\nimport sys                              # Python\u5b9f\u884c\u74b0\u5883\u306b\u95a2\u3059\u308b\u60c5\u5831\u3092\u6271\u3046\u30e9\u30a4\u30d6\u30e9\u30ea\r\nfrom multiprocessing import Pool        # \u8907\u6570\u306e\u30d7\u30ed\u30bb\u30b9\u3067\u4e26\u5217\u51e6\u7406\u3092\u884c\u3046\u305f\u3081\u306e\u30e9\u30a4\u30d6\u30e9\u30ea\r\nfrom itertools import product           # \u8907\u6570\u30ea\u30b9\u30c8\u306e\u5168\u7d44\u5408\u305b\u3092\u751f\u6210\u3059\u308b\u305f\u3081\u306e\u30e9\u30a4\u30d6\u30e9\u30ea\r\nfrom tqdm import tqdm                   # \u9032\u6357\u30d0\u30fc\uff08\u51e6\u7406\u306e\u9032\u884c\u5ea6\uff09\u3092\u8868\u793a\u3059\u308b\u30e9\u30a4\u30d6\u30e9\u30ea\r\nfrom datetime import datetime           # \u73fe\u5728\u65e5\u6642\u3092\u53d6\u5f97\u3059\u308b\u305f\u3081\u306e\u30e9\u30a4\u30d6\u30e9\u30ea\r\n\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\n# \u5404\u30d7\u30ed\u30bb\u30b9\uff08\u4e26\u5217\u51e6\u7406\uff09\u3067\u5b9f\u884c\u3059\u308b\u95a2\u6570\r\ndef run_main_CvC(params):\r\n    #---------------------------------------------------------------------------\r\n    uid, MIA, MIB = params              # \u5f15\u6570\u304b\u3089 uid, MIA, MIB\u306e\u5024\u3092\u53d6\u308a\u51fa\u3059\r\n    nGame = 100                         # \u5bfe\u6226\u6570\r\n    #---------------------------------------------------------------------------\r\n    # \u5916\u90e8\u30d7\u30ed\u30b0\u30e9\u30e0 main_CvC_MCTS.py \u3092\u547c\u3073\u51fa\u3059\r\n    subprocess.run([\r\n        sys.executable, "main_CvC_MCTS.py",     # sys.executable \u3067\u5b9f\u884c\u4e2d\u306ePython\u3092\u4f7f\u7528\r\n        "-n",    "100",                         # \u5bfe\u6226\u56de\u6570\u3092100\u56de\u306b\u8a2d\u5b9a\r\n        "--miA", str(MIA),                      # MCTS(A)\u306e iteration\u56de\u6570\uff08\u5927\u304d\u3044\u307b\u3069\u63a2\u7d22\u304c\u6df1\u3044\uff09\r\n        "--miB", str(MIB),                      # MCTS(B)\u306e iteration\u56de\u6570\uff08\u5927\u304d\u3044\u307b\u3069\u63a2\u7d22\u304c\u6df1\u3044\uff09\r\n        "--rh",  f"MCTS_{MIA}_vs_{MIB}_x{nGame}_{uid:010}_History"      # \u51fa\u529b\u3059\u308b\u5c65\u6b74\u30d5\u30a1\u30a4\u30eb\u540d\u3092\u6307\u5b9a\r\n    ])\r\n\r\n#\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\r\n# \u30e1\u30a4\u30f3\u51e6\u7406\u90e8\u5206\r\nif __name__ == "__main__":\r\n    process_count = 10                  # \u4e26\u5217\u3067\u52d5\u304b\u3059\u30d7\u30ed\u30bb\u30b9\u306e\u6570\uff08CPU\u30b3\u30a2\u6570\u306b\u5408\u308f\u305b\u3066\u8abf\u6574\u53ef\uff09\r\n    #---------------------------------------------------------------------------\r\n    # MIA\u3068MIB\u306e\u5024\u3092100\u304b\u30891000\u307e\u3067\u3001100\u3054\u3068\u306b\u5897\u52a0\u3055\u305b\u305f\u30ea\u30b9\u30c8\u3092\u4f5c\u308b\r\n    MIA_values = range(100, 1001, 100)\r\n    MIB_values = range(100, 1001, 100)\r\n    #---------------------------------------------------------------------------\r\n    # MIA\u3068MIB\u306e\u5168\u3066\u306e\u7d44\u5408\u305b\uff08\u30da\u30a2\uff09\u3092\u4f5c\u6210\uff08\u4f8b: (100,100), (100,200), ...\uff09\r\n    param_combinations   = list(product(MIA_values, MIB_values))\r\n    total_combinations   = len(param_combinations)          # \u7d44\u5408\u305b\u7dcf\u6570\uff0810\u00d710=100\uff09\r\n    indexed_combinations = [(i, mia, mib) for i, (mia, mib) in enumerate(param_combinations)]\r\n    #---------------------------------------------------------------------------\r\n    # \u958b\u59cb\u6642\u523b\u3092\u8868\u793a\r\n    start_time = datetime.now()\r\n    print(f"\u5b9f\u884c\u958b\u59cb\u6642\u523b: {start_time.strftime('%Y-%m-%d %H:%M:%S')}")\r\n    #---------------------------------------------------------------------------\r\n    # \u30d7\u30ed\u30bb\u30b9\u30d7\u30fc\u30eb\uff08\u8907\u6570\u30d7\u30ed\u30bb\u30b9\uff09\u3092\u4f5c\u6210\r\n    with Pool(process_count) as pool:\r\n        #-----------------------------------------------------------------------\r\n        # imap_unordered\u3092\u4f7f\u3044\u3001param_combinations\u306e\u5404\u8981\u7d20\u3092\u4e26\u5217\u5b9f\u884c\r\n        # \u4e26\u884c\u3057\u3066\u51e6\u7406\u304c\u9032\u307f\u3001\u51e6\u7406\u7d50\u679c\u3092\u9806\u4e0d\u540c\u3067\u53d6\u5f97\u3067\u304d\u308b\u30a4\u30c6\u30ec\u30fc\u30bf\u304c\u8fd4\u308b\r\n        results_iterator = pool.imap_unordered(run_main_CvC, indexed_combinations)\r\n        #-----------------------------------------------------------------------\r\n        # tqdm\u3067\u9032\u6357\u30d0\u30fc\u3092\u8868\u793a\uff08\u30a4\u30c6\u30ec\u30fc\u30bf\u3092\u9032\u3081\u308b\u305f\u3073\u306b\u9032\u884c\u5ea6\u304c\u66f4\u65b0\u3055\u308c\u308b\uff09\r\n        for _ in tqdm(results_iterator, total=total_combinations):\r\n            pass\r\n    #---------------------------------------------------------------------------\r\n    # \u7d42\u4e86\u6642\u523b\u3092\u8868\u793a\r\n    end_time = datetime.now()\r\n    print(f"\u5b9f\u884c\u7d42\u4e86\u6642\u523b: {end_time.strftime('%Y-%m-%d %H:%M:%S')}")\r\n    #---------------------------------------------------------------------------\r\n    # \u958b\u59cb\u304b\u3089\u7d42\u4e86\u307e\u3067\u306e\u51e6\u7406\u6642\u9593\u3092\u8a08\u7b97\u3001\u8868\u793a\u3059\u308b\u3002\r\n    duration = (end_time - start_time).total_seconds()\r\n    print(f"\u5b9f\u884c\u6642\u9593: {duration:.1f} [sec]")\r\n<\/pre>\n
3) \u3044\u3056\u5b9f\u884c<\/h2>\n
100\u30bb\u30c3\u30c8 x 100\u8a66\u5408\u306e\u5bfe\u5c40\u30c7\u30fc\u30bf\u306e\u53ce\u96c6\u306b\u3001\u6211\u304c\u5bb6\u306e\u6700\u65b0\u30de\u30b7\u30f3\u3067\u7d04 3.7\u6642\u9593<\/span> \u3092\u8981\u3057\u305f\u3002
\n\u5b9f\u884c\u74b0\u5883\u306e\u30de\u30b7\u30f3\u30b9\u30da\u30c3\u30af\u306f\u4e0b\u8a18\u306e\u901a\u308a\u3002
\n\n
Parts<\/th>Name\/Spec<\/th><\/tr>
CPU<\/td>Intel Core Ultra7 265KF 20cores<\/td><\/tr>
Memory<\/td>DDR5-5600 96GB<\/td><\/tr>
Motherboard<\/td>Z890<\/td><\/tr>
SSD<\/td>M.2 NVMe PCIe4.0 2TB<\/td><\/tr>
\u30b0\u30e9\u30dc<\/td>\u4eca\u56de\u306f CPU\u5b9f\u884c\u306a\u306e\u3067\u4f7f\u7528\u305b\u305a<\/td><\/tr>\n<\/table>
\n\"\"<\/p>\n
Bash shell\u4e0a\u3067\u30d7\u30ed\u30b0\u30e9\u30e0\u3092\u5b9f\u884c\u3059\u308b\u3002
\nASUS\u88fd\u30de\u30b6\u30fc\u30dc\u30fc\u30c9\u306e CPU\u8b66\u544a LED\u304c\u8d64\u304f\u70b9\u706f\u3059\u308b\u307b\u3069\u3001\u9ad8\u6e29\u306b\u306a\u3063\u305f\u30fb\u30fb\u30fb<\/span><\/p>\n
\r\n$ python main_Collect_MCTS_training_data.py<\/span>\r\n\u5b9f\u884c\u958b\u59cb\u6642\u523b: 2025-06-04 11:38:15\r\n100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 100\/100 [3:40:04<00:00, 132.05s\/it]\r\n\u5b9f\u884c\u7d42\u4e86\u6642\u523b: 2025-06-04 15:18:19\r\n\u5b9f\u884c\u6642\u9593: 13204 [sec]\r\n<\/pre>\n
\u53ce\u96c6\u3067\u304d\u305f\u8a13\u7df4\u30c7\u30fc\u30bf\u306f\u5168200\u30d5\u30a1\u30a4\u30eb\u3060\u3002
\n\u63a2\u7d22\u56de\u6570\u306e\u7d44\u5408\u305b 100\u30bb\u30c3\u30c8 x 2\uff08\u5148\u624b\u30fb\u5f8c\u624b\uff09= 200\u30d5\u30a1\u30a4\u30eb<\/p>\n
4) \u30c7\u30fc\u30bf\u306e\u4e2d\u8eab\u3092\u78ba\u8a8d<\/h2>\n
\u7121\u4f5c\u70ba\u306b 1\u8a66\u5408\u306e\u30c7\u30fc\u30bf\u3092\u9078\u629e\u3057\u3001\u4e2d\u8eab\u3092\u8868\u793a\u3057\u3066\u307f\u308b\u3002
\n\u4eca\u56de\u305f\u307e\u305f\u307e\u9078\u3093\u3060\u306e\u306f\u3001
\nMCTS\uff08\u63a2\u7d22\u56de\u6570900\u56de\uff09vs MCTS\uff08\u63a2\u7d22\u56de\u6570900\u56de\uff09<\/span>
\n\u306e\u7d50\u679c\u3060\u3002 <\/p>\n
\u5148\u624b\u3001\u5f8c\u624b\u306e\u30c7\u30fc\u30bf\u30d5\u30a1\u30a4\u30eb\u3092\u30ed\u30fc\u30c9\u3059\u308b\u3002
\n\u5404\u30d5\u30a1\u30a4\u30eb\u306e\u4e2d\u306b\u306f 100\u8a66\u5408\u5206\u306e\u6307\u3057\u624b\u30c7\u30fc\u30bf\uff08\u5404\u7d041600\u624b\u5206\uff09\u304c\u76f4\u5217\u306b\u683c\u7d0d\u3055\u308c\u3066\u3044\u308b\u3002
\n
\n>>> import pickle\n>>>\n>>> filename1 = \"MCTS_900_vs_900_x100_0000000088_History_P1_2025_0604_144656.pth\"\n>>> with open(filename1, \"rb\") as f:\n...     data1 = pickle.load(f)\n>>>\n>>> filename2 = \"MCTS_900_vs_900_x100_0000000088_History_P2_2025_0604_144656.pth\"\n>>> with open(filename2, \"rb\") as f:\n...     data2 = pickle.load(f)\n>>>\n>>> len(data1)\n1617\n>>> len(data2)\n1583\n<\/pre>
\n\u5148\u624b\u306e\u6307\u3057\u624b\u30c7\u30fc\u30bf\u306f 1\u30d5\u30a1\u30a4\u30eb\u5185\u306b 100\u8a66\u5408\u3067\u5168 1617\u624b\u5206\u3001
\n\u5f8c\u624b\u306e\u6307\u3057\u624b\u30c7\u30fc\u30bf\u306f 1\u30d5\u30a1\u30a4\u30eb\u5185\u306b 100\u8a66\u5408\u3067\u5168 1583\u624b\u5206\u304c\u5b58\u5728\u3059\u308b\u3002
\n\u4e21\u8005\u306e\u624b\u6570\u304c\u9055\u3046\u306e\u306f\u3001\u30d1\u30b9\u306b\u3088\u308a\u624b\u756a\u3092\u30b9\u30ad\u30c3\u30d7\u3055\u308c\u308b\u5834\u5408\u304c\u3042\u308b\u305f\u3081\u3002<\/p>\n
\u5148\u624b\u306e 1\u8a66\u5408\u76ee\u306e \u7b2c\u4e00\u624b\u3092\u8868\u793a\u3057\u3066\u307f\u308b\u3002
\n
\n>>> from pprint import pprint\n>>> pprint(data1[0])\n(array([[ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0, -1,  1,  0,  0],\n       [ 0,  0,  1, -1,  0,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0]], dtype=int8),\n 22,\n array([[ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0, -1,  1,  0,  0],\n       [ 0,  0,  1,  1,  1,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0]], dtype=int8),\n np.float64(0.35000000000000003),\n False)\n<\/pre><\/p>\n
\u5f8c\u624b\u306e 1\u8a66\u5408\u76ee\u306e \u7b2c\u4e00\u624b\u3092\u8868\u793a\u3057\u3066\u307f\u308b\u3002
\n\u77f3\u3092\u8868\u3059 (1, -1) \u306f\u4ee5\u4e0b\u306e\u901a\u308a\u3002
\n\n
1<\/td>\u81ea\u5206\u306e\u77f3<\/td><\/tr>
-1<\/td>\u76f8\u624b\u306e\u77f3<\/td><\/tr>\n<\/table>
\n\u3053\u306e\u305f\u3081\u3001\u624b\u756a\u304c\u5909\u308f\u308c\u3070\u77f3\u306e\u8868\u73fe\u3082\u6b63\u8ca0\u304c\u53cd\u8ee2<\/span> \u3059\u308b\u3053\u3068\u306b\u6ce8\u610f\u3059\u308b\u3002
\n
\n>>> pprint(data2[0])\n(array([[ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  1, -1,  0,  0],\n       [ 0,  0, -1, -1, -1,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0]], dtype=int8),\n 26,\n array([[ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0],\n       [ 0,  0,  1, -1,  0,  0],\n       [ 0,  0,  1, -1, -1,  0],\n       [ 0,  0,  1,  0,  0,  0],\n       [ 0,  0,  0,  0,  0,  0]], dtype=int8),\n np.float64(0.2),\n False)\n<\/pre><\/p>\n
\u76e4\u4e0a\u306e\u77f3\u306e\u914d\u7f6e\u304c 1, -1 \u3060\u3068\u898b\u3065\u3089\u3044\u306e\u3067\u3001o x \u306b\u5909\u3048\u3066\u307f\u308b\u3002
\n
\n>>> def convert_and_print(board):\n...     symbol_map = {\n...         -1: 'x',\n...          0: '.',\n...          1: 'o'\n...     }\n...     for row in board:\n...         print(' '.join(symbol_map[int(cell)] for cell in row))\n...\n>>>\n>>> convert_and_print(data1[0][0])    # \u5148\u624b1\u624b\u76ee\u3001\u6253\u3064\u524d<\/span>\n. . . . . .\n. . . . . .\n. . x o . .\n. . o x . .\n. . . . . .\n. . . . . .\n>>> convert_and_print(data1[0][2])    # \u5148\u624b1\u624b\u76ee\u3001\u6253\u3063\u305f\u5f8c<\/span>\n. . . . . .\n. . . . . .\n. . x o . .\n. . o o o .\n. . . . . .\n. . . . . .\n>>>\n>>> convert_and_print(data2[0][0])    # \u5f8c\u624b1\u624b\u76ee\u3001\u6253\u3064\u524d \u203b\u81ea\u5206\u8996\u70b9\u306b\u6b63\u8ca0\u53cd\u8ee2\u6e08\u307f<\/span>\n. . . . . .\n. . . . . .\n. . o x . .\n. . x x x .\n. . . . . .\n. . . . . .\n>>> convert_and_print(data2[0][2])    # \u5f8c\u624b1\u624b\u76ee\u3001\u6253\u3063\u305f\u5f8c<\/span>\n. . . . . .\n. . . . . .\n. . o x . .\n. . o x x .\n. . o . . .\n. . . . . .\n<\/pre><\/p>\n
\u6b21\u306b\u3001\u52dd\u6557\u304c\u6c7a\u3057\u305f\u5834\u9762\u3092\u898b\u3066\u307f\u308b\u3002
\n\u7b2c\u4e00\u8a66\u5408\u306f\u5148\u624b\u30fb\u5f8c\u624b\u305d\u308c\u305e\u308c 15\u624b\u76ee\u3067\u8a66\u5408\u7d42\u4e86\u3057\u305f\u306e\u3067\u3001\u3053\u306e\u5834\u9762\u3092\u898b\u3066\u307f\u308b\u3002
\n
\n>>> pprint(data1[15])           # \u5148\u624b\u306e 15\u624b\u76ee<\/span>\n(array([[ 1,  1,  1,  1,  1,  1],\n       [-1, -1, -1, -1, -1,  0],\n       [-1, -1, -1, -1, -1,  1],\n       [ 1,  1, -1,  1, -1,  1],\n       [ 0,  1, -1, -1,  1,  1],\n       [-1,  1,  1,  1,  1,  1]], dtype=int8),\n 11,\n array([[ 1,  1,  1,  1,  1,  1],\n       [-1, -1, -1, -1, -1,  1],\n       [-1, -1, -1, -1,  1,  1],\n       [ 1,  1, -1,  1, -1,  1],\n       [ 0,  1, -1, -1,  1,  1],\n       [-1,  1,  1,  1,  1,  1]], dtype=int8),\n np.float64(1.05),\n True)\n>>> pprint(data2[15])           # \u5f8c\u624b\u306e 15\u624b\u76ee<\/span>\n(array([[-1, -1, -1, -1, -1, -1],\n       [ 1,  1,  1,  1,  1, -1],\n       [ 1,  1,  1,  1, -1, -1],\n       [-1, -1,  1, -1,  1, -1],\n       [ 0, -1,  1,  1, -1, -1],\n       [ 1, -1, -1, -1, -1, -1]], dtype=int8),\n 24,\n array([[-1, -1, -1, -1, -1, -1],\n       [ 1,  1,  1,  1,  1, -1],\n       [ 1,  1,  1,  1, -1, -1],\n       [ 1,  1,  1, -1,  1, -1],\n       [ 1,  1,  1,  1, -1, -1],\n       [ 1, -1, -1, -1, -1, -1]], dtype=int8),\n np.float64(1.6),\n True)\n>>>\n>>> data1[15][4]      # \u5148\u624b\u306e\u7d42\u4e86\u30d5\u30e9\u30b0\u304c True\u306b\u306a\u3063\u3066\u3044\u308b\u3002<\/span>\nTrue\n>>> data2[15][4]      # \u5f8c\u624b\u306e\u7d42\u4e86\u30d5\u30e9\u30b0\u304c True\u306b\u306a\u3063\u3066\u3044\u308b\u3002<\/span>\nTrue\n>>>\n>>> data1[15][3]      # \u5148\u624b\u306e\u3053\u306e\u4e00\u624b\u306e\u5831\u916c\u306f 1.05 \u203b\u3053\u306e\u8a66\u5408\u306f\u5f15\u304d\u5206\u3051\u3060\u3063\u305f\u3088\u3046\u3060\u3002<\/span>\nnp.float64(1.05)\n>>> data2[15][3]      # \u5f8c\u624b\u306e\u3053\u306e\u4e00\u624b\u306e\u5831\u916c\u306f 1.6<\/span>\nnp.float64(1.6)\n<\/pre><\/p>\n
\u3053\u306e\u5834\u9762\u3082\u308f\u304b\u308a\u3084\u3059\u304f\u8868\u793a\u3057\u3066\u307f\u308b\u3002
\n\u78ba\u304b\u306b 18 - 18 \u306e\u5f15\u304d\u5206\u3051\u3067\u7d42\u308f\u3063\u3066\u3044\u308b\u3002
\n
\n>>> convert_and_print(data1[15][2])      # \u5148\u624b\u304c\u6700\u5f8c\u306e\u4e00\u624b\u3092\u6253\u3063\u305f\u76f4\u5f8c<\/span>\no o o o o o\nx x x x x o\nx x x x o o\no o x o x o\n. o x x o o\nx o o o o o\n>>> convert_and_print(data2[15][0])      # \u5f8c\u624b\u304c\u6700\u5f8c\u306e\u4e00\u624b\u3092\u6253\u3064\u76f4\u524d \u203b\u4e0a\u8a18\u76e4\u9762\u3092\u81ea\u5206\u8996\u70b9\u306b\u53cd\u8ee2\u6e08\u307f<\/span>\nx x x x x x\no o o o o x\no o o o x x\nx x o x o x\n. x o o x x\no x x x x x\n>>> convert_and_print(data2[15][2])      # \u5f8c\u624b\u304c\u6700\u5f8c\u306e\u4e00\u624b\u3092\u6253\u3063\u305f\u76f4\u5f8c \u2192 \u6700\u7d42\u7684\u306a\u76e4\u9762\u306e\u72b6\u614b<\/span>\nx x x x x x\no o o o o x\no o o o x x\no o o x o x\no o o o x x\no x x x x x\n<\/pre><\/p>\n
\u30103\u3011\u4eca\u5f8c\u306e\u4e88\u5b9a<\/h1>\n
\u73fe\u6642\u70b9\u307e\u3067\u306b\u51fa\u6765\u305f\u3053\u3068\uff1a
\nMCTS vs MCTS\u306e\u5f37\u5f31100\u901a\u308a\u306e\u7d44\u307f\u5408\u308f\u305b\u3067 100\u30bb\u30c3\u30c8 x 100\u8a66\u5408\u5206\u306e\u6307\u3057\u624b\u5c65\u6b74\u30c7\u30fc\u30bf\u304c\u53ce\u96c6\u3067\u304d\u305f\u3002
\n1\u8a66\u5408\u3042\u305f\u308a\u306e\u6307\u3057\u624b\u6570\u304c\u3001\u5148\u624b\u30fb\u5f8c\u624b\u5408\u308f\u305b\u3066\u7d04 32\u306a\u306e\u3067\u3001
\n100\u30bb\u30c3\u30c8 x 100\u8a66\u5408 x 32\u624b = 320,000 \u624b\u5206\u306e\u8a13\u7df4\u30c7\u30fc\u30bf\u3092\u4f5c\u6210\u3067\u304d\u305f\u3002<\/span><\/p>\n
\u7d1b\u5931\u3059\u308b\u3068\u56f0\u308b\u306e\u3067\u3001\u3053\u3053\u306b\u3082\u7f6e\u3044\u3066\u304a\u3053\u3046\u3002
\ntraining_data_20250604.zip<\/a><\/p>\n
\u6b21\u306b\u3084\u308b\u3053\u3068\uff1a
\n\u3053\u306e\u30c7\u30fc\u30bf\u3092 DQN\u306e\u6559\u5e2b\u30c7\u30fc\u30bf\u306b\u4f7f\u3044\u3001DQN\u30aa\u30bb\u30ed AI\u3092\u8a13\u7df4\u3059\u308b\u3002<\/span><\/p>\n
\u679c\u305f\u3057\u3066 MCTS\u30ec\u30d9\u30eb\u306e\u5f37\u3044\u30aa\u30bb\u30edAI\u306b\u80b2\u3064\u306e\u304b\uff1f<\/span>
\n\u3092\u5b9f\u9a13\u3057\u3066\u3044\u304d\u305f\u3044\u3002<\/p>\n
\u4f5c\u3063\u305f\u30c7\u30fc\u30bf\u306e\u30af\u30ec\u30f3\u30b8\u30f3\u30b0\u3084\u30c7\u30fc\u30bf\u62e1\u5f35\u306b\u3088\u308b\u5897\u6b96\u306b\u3064\u3044\u3066\u3082\u8981\u691c\u8a0e\u3060\u304c\u3001\u6700\u521d\u306f\u4eba\u9593\u5224\u65ad\u306e\u6700\u9069\u5316\u3092\u305b\u305a\u306b\u3084\u3063\u3066\u307f\u308b\u4e88\u5b9a\u3060\u3002
\n\u56de\u8ee2\u306b\u3088\u308b\u30c7\u30fc\u30bf 4\u500d\u62e1\u5f35\u3050\u3089\u3044\u306f\u3084\u308d\u3046\u304b\u306a\u30fb\u30fb\u30fb<\/p>\n
\n","protected":false},"excerpt":{"rendered":"
\u30101\u3011\u3084\u308a\u305f\u3044\u3053\u3068 \u524d\u56de\u306e\u6295\u7a3f\u3067\u6848\u51fa\u3057\u3057\u305f\u5185\u5bb9\u3067\u3001\u5b9f\u88c5\u3092\u9032\u3081\u305f\u3044\u3002 (53)\u3010Othello AI\u3011\u30e2\u30f3\u30c6\u30ab\u30eb\u30ed\u6cd5\u3067 DQN\u306e\u6559\u5e2b\u30c7\u30fc\u30bf\u3092\u4f5c\u308d\u3046\u304b\uff1f DQN\u65b9\u5f0f\u30aa\u30bb\u30edAI\u306e\u8a13\u7df4\u30c7\u30fc\u30bf\u3068\u3057\u3066\u3001 \u30e2\u30f3\u30c6\u30ab\u30eb\u30ed\u63a2\u7d22\u65b9\u5f0f\u30aa\u30bb\u30edA\u2026 \u7d9a\u304d\u3092\u8aad\u3080 »<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[28,6,31,29],"tags":[],"class_list":["post-2293","post","type-post","status-publish","format-standard","hentry","category-othelloai","category-6","category-31","category-29"],"views":1079,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/posts\/2293","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/comments?post=2293"}],"version-history":[{"count":76,"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/posts\/2293\/revisions"}],"predecessor-version":[{"id":2532,"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/posts\/2293\/revisions\/2532"}],"wp:attachment":[{"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/media?parent=2293"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/categories?post=2293"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dogrow.net\/nnet\/wp-json\/wp\/v2\/tags?post=2293"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}