bluemixのRetrieve and Rankを試す(curlでチュートリアル)
デザインがちょっと変わったので全部コマンドラインでやる
なんかバグっぽい挙動していてうまく動かないんだもん(´・ω・`)
最後に全部削除しています。
サービス作成
サービス一覧を確認
cf marketplace cf marketplace -s retrieve_and_rank
サービスを作成
cf create-service retrieve_and_rank standard rr_tutorial_01
資格情報を作成
cf create-service-key rr_tutorial_01 rr_tutorial_01-credentials
資格情報を確認
cf service-key rr_tutorial_01 rr_tutorial_01-credentials
↓
{ "password": "7kpbrZJ4n1Cq", "url": "https://gateway.watsonplatform.net/retrieve-and-rank/api", "username": "d2affa4a-a293-4bba-98c4-b03501299e66" }
クラスター作成
クラスター作成
curl -X POST -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters" -d ""
↓
{ "solr_cluster_id":"sc8ac937f6_4cc5_44a2_98b3_d249c853391f", "cluster_name":"", "cluster_size":"", "solr_cluster_status":"NOT_AVAILABLE" }
クラスター作成を確認
curl -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f"
↓
{ "solr_cluster_id":"sc8ac937f6_4cc5_44a2_98b3_d249c853391f", "cluster_name":"", "cluster_size":"", "solr_cluster_status":"READY" }
※READYがあること
コンフィグファイルのサンプルをダウンロード
wget https://github.com/watson-developer-cloud/doc-tutorial-downloads/raw/master/retrieve-and-rank/cranfield-solr-config.zip
※今回はサンプル
コンフィグファイルをアップロード
curl -X POST -H "Content-Type: application/zip" \ -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f/config/example_config" \ --data-binary @cranfield-solr-config.zip
↓
{ "message":"WRRCSR026: Successfully uploaded named config [example_config] for Solr cluster [sc8ac937f6_4cc5_44a2_98b3_d249c853391f].", "statusCode":200 }
コンフィグファイルを関連付け
curl -X POST \ -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f/solr/admin/collections" \ -d "action=CREATE&name=example_collection&collection.configName=example_config"
↓
<?xml version="1.0" encoding="UTF-8"?> <response> <lst name="responseHeader"> <int name="status">0</int> <int name="QTime">12332</int> </lst> <lst name="success"> <lst name="10.176.142.253:6173_solr"> <lst name="responseHeader"> <int name="status">0</int> <int name="QTime">2541</int> </lst> <str name="core">example_collection_shard1_replica2</str> </lst> <lst name="10.176.39.48:5590_solr"> <lst name="responseHeader"> <int name="status">0</int> <int name="QTime">3026</int> </lst> <str name="core">example_collection_shard1_replica1</str> </lst> </lst> </response>
ドキュメントファイル用のサンプルをダウンロード
wget https://github.com/watson-developer-cloud/doc-tutorial-downloads/raw/master/retrieve-and-rank/cranfield-data.json
ドキュメントファイルのアップロード
curl -X POST -H "Content-Type: application/json" \ -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f/solr/example_collection/update" \ --data-binary @cranfield-data.json
↓
{ "responseHeader":{ "status":0, "QTime":1673 } }
ランカー用設定
グランドトゥルースファイルのサンプルをダウンロード
wget https://github.com/watson-developer-cloud/doc-tutorial-downloads/raw/master/retrieve-and-rank/cranfield-gt.csv
トレーニング用pythonスクリプトをダウンロード
wget https://github.com/watson-developer-cloud/doc-tutorial-downloads/raw/master/retrieve-and-rank/train.py
トレーニングスクリプト実施
python ./train.py -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ -i cranfield-gt.csv \ -c sc8ac937f6_4cc5_44a2_98b3_d249c853391f \ -x example_collection \ -n "example_ranker"
↓
Input file is cranfield-gt.csv Solr cluster is sc8ac937f6_4cc5_44a2_98b3_d249c853391f Solr collection is example_collection Ranker name is example_ranker Rows per query 10 Generating training data... Generating training data complete. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 304k 0 319 100 304k 24 23456 0:00:13 0:00:13 --:--:-- 17948 { "ranker_id":"c852bax18-rank-2794", "name":"example_ranker", "created":"2016-10-24T13:47:07.199Z", "url":"https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/rankers/c852bax18-rank-2794", "status":"Training", "status_description":"The ranker instance is in its training phase, not yet ready to accept rank requests" }
Solrの検索クエリをブラウザで確認
https://d2affa4a-a293-4bba-98c4-b03501299e66:7kpbrZJ4n1Cq@gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f/solr/example_collection/select?q=what is the basic mechanism of the transonic aileron buzz&wt=json&fl=id,title
↓
"responseHeader":{ "status":0, "QTime":16, "params":{ "q":"what is the basic mechanism of the transonic aileron buzz", "fl":"id,title", "wt":"json" } }, "response":{ "numFound":164, "start":0, "docs":[ { "id":"496", "title":[ "a theory of transonic aileron buzz, neglecting viscous effects ." ] }, { "id":"1268", "title":[ "stable combustion of a high-velocity gas in a heated boundary layer ." ] },{ "id":"903", "title":[ "two dimensional transonic unsteady flow with shock waves ." ] },{ "id":"520", "title":[ "wing-tail interference as a cause of 'magnus' effects on a finned missile ." ] },{ "id":"199", "title":[ "measurement of two dimensional derivatives on a wing-aileron-tab system ." ] },{ "id":"643", "title":[ "an investigation of wing-aileron flutter using ground launched rocket models ." ] },{ "id":"1072", "title":[ "ignition and combustion in a laminar mixing zone ." ] },{ "id":"262", "title":[ "the formation of a blast wave by a very intense explosion ." ] },{ "id":"1248", "title":[ "an analytic extension of the shock-expansion method ." ] },{ "id":"660", "title":[ "the fundamental solution for small steady three dimensional disturbances to a two dimensional parallel shear flow ." ] } ] } }
ランカーの作成を確認
curl -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/rankers/c852bax18-rank-2794"
↓
{ "ranker_id":"c852bax18-rank-2794", "name":"example_ranker", "created":"2016-10-24T13:47:07.199Z", "url":"https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/rankers/c852bax18-rank-2794", "status":"Available", "status_description":"The ranker instance is now available and is ready to take ranker requests." }
ランク結果をブラウザで確認
https://d2affa4a-a293-4bba-98c4-b03501299e66:7kpbrZJ4n1Cq@gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f/solr/example_collection/fcselect?ranker_id=c852bax18-rank-2794&q=what is the basic mechanism of the transonic aileron buzz&wt=json&fl=id,title
↓
{ "responseHeader":{ "status":0, "QTime":679 }, "response":{ "numFound":164, "start":0, "maxScore":10.0, "docs":[ { "id":"313", "title":[ "on alternative forms for the basic equations of transonic flow theory ." ] },{ "id":"496", "title":[ "a theory of transonic aileron buzz, neglecting viscous effects ." ] },{ "id":"520", "title":[ "wing-tail interference as a cause of 'magnus' effects on a finned missile ." ] },{ "id":"903", "title":[ "two dimensional transonic unsteady flow with shock waves ." ] },{ "id":"1268", "title":[ "stable combustion of a high-velocity gas in a heated boundary layer ." ] },{ "id":"1072", "title":[ "ignition and combustion in a laminar mixing zone ." ] },{ "id":"262", "title":[ "the formation of a blast wave by a very intense explosion ." ] },{ "id":"643", "title":[ "an investigation of wing-aileron flutter using ground launched rocket models ." ] },{ "id":"199", "title":[ "measurement of two dimensional derivatives on a wing-aileron-tab system ." ] },{ "id":"1242", "title":[ "some considerations on the laminar stability of time-dependent basic flows ." ] } ] } }
削除
クラスター削除
curl -i -X DELETE \ -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc8ac937f6_4cc5_44a2_98b3_d249c853391f"
↓
HTTP/1.1 200 OK X-Backside-Transport: OK OK Connection: Keep-Alive Transfer-Encoding: chunked Date: Mon, 24 Oct 2016 14:43:16 GMT Content-Type: application/json Server: - Set-Cookie: Watson-DPAT=OQsYXi8uoqg6kh5E4EU2qrm9VwsHdb%2BiXd%2Fw4%2FddxcYCDDr54ieiie7IqLO4VCOpn%2BjFquCI6Qx%2BNnPQirRQjwUjQ76mGx1bNC%2F30R75gMzM7DH%2BVtn0C8awkB3Q%2FmbrpAtq78TdEXuq9PYewL11w1q1H5Pun8HIVnDZRUGIvTtAyLMYKtLU78puiczdvcAd2os68%2FuWnBUPsnXbAHe03VEvLK6TzadSseoyjzjmcVdgutrRpt1HcR983aY%2FYnpr8I7WkrTNNpHOVX%2FriCxD8bjorCwTuuEZR87DWYK7UEneB%2Bpe6jKXhnftRZcoezBv66xtpJq07%2FdCswBxKLgeH9PKtAUggYCqMOyn3vfEUdTyo0iD66P901Qg5gI9mWtE8MQfTbhWcsStpcWpN6R%2FnCrd5cKcvCxfT3qK4BzYi8ZbEsKAr3TLud504FM%2FZxfKIT8PY%2B%2BzWE%2B1HmvHFGozUQucznWgn0ZDEBcx%2BWwezYlcQrlfNtJfz6V7D%2BC%2BmUnP97BTfItDwYbJ0IQtDRaNj0or1jW%2FpxRfve7ZrmokSjzm%2F9PEXyHEELxHu5rW3hL%2Bpr8zF2i9vkusu8q40Jg%2BSwlny1tZpfpoiGtZjm8mMjkOW2Rx6iVd7v4KhEpgXaOqBBBqXI1Vnpq8b8yM1daJbpprza9ufrHvJFbBWsvqJs1x1pqoRlmPHGUvraRNCzw5v3Y44UZez8dMRrS18qWNTLuTtnk1dQIQBEpttRj%2FvmG1cQnbP8F5FRLIgs%2B4PoLFmvM21Jik7Gq9O2Kyfq5tbSIBUiV6%2BiLheafrHt%2Fhx9Ql5dMuXSgU5E0beWfh%2FLs3bKI9W8QFklA5gC03UJ7DtYljp4HS%2FTqbzSDF2OJV00Xs4Q1E%2F9RTd62b9HfPTMUSq2eHZBzoaivr41Ls8%2FjgebwILaO3Dvn5cFqKZqtxDbN3HwJsHiMmPWtyHE%2FLDbznhMorRaXMoR8vi0tKV9AdELroSVUwJe5b3DmlZughvrJy07WflTLtug%3D%3D; path=/retrieve-and-rank/api; secure; HttpOnly X-Client-IP: 119.104.7.50 X-Global-Transaction-ID: 51546924 X-DP-Watson-Tran-ID: gateway-dp01-51546924 { "message":"WRRCSR023: Successfully deleted Solr cluster [sc8ac937f6_4cc5_44a2_98b3_d249c853391f].", "statusCode":200 }
ランカー削除
curl -X DELETE \ -u "d2affa4a-a293-4bba-98c4-b03501299e66":"7kpbrZJ4n1Cq" \ "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/rankers/c852bax18-rank-2794"
↓
{}
サービス削除
cf delete-service-key rr_tutorial_01 rr_tutorial_01-credentials -f cf delete-service rr_tutorial_01 -f
所感
今回はどのようなデータを用意すればよいのか不明だったのでチュートリアルをそのままやってみた。
質問に対して回答リストを取得している感じ、まだ、ランカーの順位付けや日本語対応などはわかっていないので
少しずつ調べていく。