Elasticsearch(四)索引一亿条订单数据


创建索引和映射

PUT orderlist
{
    "settings": {
        "index": {
            "number_of_shards": 6,
            "number_of_replicas": 1
        }
    },
    "mappings": {
        "properties": {
          "order_sn": {
            "type": "keyword"
          },
          "order_type": {
            "type": "byte"
          },
          "total_amount": {
            "type": "short"
          },
          "pay_amount": {
            "type": "short"
          },
          "coupon_amount": {
            "type": "short"
          },
          "status": {
              "type": "byte"
          },
          "created": {
              "type": "integer"
          },
          "partner_id": {
              "type": "short"
          },
          "partner_type": {
              "type": "byte"
          },
          "entry_id": {
              "type": "short"
          },
          "entry_type": {
              "type": "byte"
          },
          "org_id": {
              "type": "integer"
          },
          "org_province": {
              "type": "keyword"
          },
          "org_city": {
              "type": "keyword"
          },
          "album_id": {
              "type": "integer"
          },
          "author_id": {
              "type": "integer"
          },
          "uid": {
              "type": "integer"
          },
          "sex": {
              "type": "byte"
          },
          "birthday": {
              "type": "date",
              "format": "yyyy-MM-dd"
          },
          "province": {
              "type": "keyword"
          },
          "city": {
              "type": "keyword"
          },
          "register_time": {
              "type": "integer"
          }
        }
    }
}

程序

数据生成使用Golang写的

使用bulk批量索引,每次索引10000条,索引10次

第一次,索引1000万条数据,用时(48.85分钟)2931秒

第二次,索引1000万条数据,用时(42.70分钟)2562秒