2018年2月10日土曜日

Solr7.0 Windowsから大きいファイルをバッチ処理・登録

PowerShellとかでsolorディレクトリに移動
スタンドアロン/Win向けにpostToolがある
example/exampledocs/post.jar



\solr-7.0.0>java -Dport=8984 -Dtype=applocation/json -Dc=collection1 -Dparams="commit=true" -jar example/exampledocs/post.jar biggestFile.json
PS C:\SE\solr-7.0.0> java -jar ./example/exampledocs/post.jar --help
SimplePostTool version 5.0.0
Usage: java [SystemProperties] -jar post.jar [-h|-] [ [...]]

Supported System Properties and their defaults:
  -Dc=
  -Durl= (overrides -Dc option if specified)
  -Ddata=files|web|args|stdin (default=files)
  -Dtype= (default=application/xml)
  -Dhost= (default: localhost)
  -Dport= (default: 8983)
  -Dbasicauth= (sets Basic Authentication credentials)
  -Dauto=yes|no (default=no)
  -Drecursive=yes|no| (default=0)
  -Ddelay= (default=0 for files, 10 for web)
  -Dfiletypes=[,,...] (default=xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log)
  -Dparams="=[&=...]" (values must be URL-encoded)
  -Dcommit=yes|no (default=yes)
  -Doptimize=yes|no (default=no)
  -Dout=yes|no (default=no)

This is a simple command line tool for POSTing raw data to a Solr port.
NOTE: Specifying the url/core/collection name is mandatory.
Data can be read from files specified as commandline args,
URLs specified as args, as raw commandline arg strings or via STDIN.
Examples:
  java -Dc=gettingstarted -jar post.jar *.xml
  java -Ddata=args -Dc=gettingstarted -jar post.jar '42'
  java -Ddata=stdin -Dc=gettingstarted -jar post.jar < hd.xml
  java -Ddata=web -Dc=gettingstarted -jar post.jar http://example.com/
  java -Dtype=text/csv -Dc=gettingstarted -jar post.jar *.csv
  java -Dtype=application/json -Dc=gettingstarted -jar post.jar *.json
  java -Durl=http://localhost:8983/solr/techproducts/update/extract -Dparams=literal.id=pdf1 -jar post.jar solr-word.pdf
  java -Dauto -Dc=gettingstarted -jar post.jar *
  java -Dauto -Dc=gettingstarted -Drecursive -jar post.jar afolder
  java -Dauto -Dc=gettingstarted -Dfiletypes=ppt,html -jar post.jar afolder
The options controlled by System Properties include the Solr
URL to POST to, the Content-Type of the data, whether a commit
or optimize should be executed, and whether the response should
be written to STDOUT. If auto=yes the tool will try to set type
automatically from file name. When posting rich documents the
file name will be propagated as "resource.name" and also used
as "literal.id". You may override these or any other request parameter
through the -Dparams property. To do a commit only, use "-" as argument.
The web mode is a simple crawler following links within domain, default delay=10s.
PS C:\SE\solr-7.0.0>

0 件のコメント:

コメントを投稿

ツイキャス新卒採用2019で遊んだ

https://saiyo2019.moi.st 内容はFizzBuzzチャレンジでした websocketでリクエスト送受信してポイントを稼いでハイスコアを競うゲーム ブロードキャスティングサービスはネットワークの超低遅延を要求されるので、そういうアレ saiyo2...