Skip to content

Commit e95eb73

Browse files
Simran-BSimran-B
authored
New consolidation default values for inverted indexes and arangosearch Views in 3.12.6 (#839)
1 parent 4cdae28 commit e95eb73

File tree

10 files changed

+140
-50
lines changed

10 files changed

+140
-50
lines changed

‎site/content/arangodb/3.12/develop/http-api/indexes/inverted.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -537,7 +537,7 @@ paths:
537537
compaction operations are governed by `consolidationIntervalMsec` and the
538538
candidates for compaction are selected via `consolidationPolicy`.
539539
type: integer
540-
default: 1000
540+
default: 5000
541541
consolidationPolicy:
542542
description: |
543543
The consolidation policy to apply for selecting which segments should be merged.
@@ -570,24 +570,24 @@ paths:
570570
Defines the value (in bytes) to treat all smaller segments as equal for
571571
consolidation selection.
572572
type: integer
573-
default: 2097152
573+
default: 25165824
574574
segmentsBytesMax:
575575
description: |
576576
The maximum allowed size of all consolidated segments in bytes.
577577
type: integer
578-
default: 5368709120
578+
default: 8589934592
579579
segmentsMax:
580580
description: |
581581
The maximum number of segments that are evaluated as candidates for
582582
consolidation.
583583
type: integer
584-
default: 10
584+
default: 200
585585
segmentsMin:
586586
description: |
587587
The minimum number of segments that are evaluated as candidates for
588588
consolidation.
589589
type: integer
590-
default: 1
590+
default: 50
591591
minScore:
592592
description: |
593593
Filter out consolidation candidates with a score less than this.

‎site/content/arangodb/3.12/develop/http-api/views/arangosearch-views.md‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -302,7 +302,7 @@ paths:
302302
compaction operations are governed by `consolidationIntervalMsec` and the
303303
candidates for compaction are selected via `consolidationPolicy`.
304304
type: integer
305-
default: 10000
305+
default: 5000
306306
consolidationPolicy:
307307
description: |
308308
The consolidation policy to apply for selecting which segments should be merged.
@@ -351,24 +351,24 @@ paths:
351351
Defines the value (in bytes) to treat all smaller segments
352352
as equal for consolidation selection.
353353
type: integer
354-
default: 2097152
354+
default: 25165824
355355
segmentsBytesMax:
356356
description: |
357357
Maximum allowed size of all consolidated segments in bytes.
358358
type: integer
359-
default: 5368709120
359+
default: 8589934592
360360
segmentsMax:
361361
description: |
362362
The maximum number of segments that are evaluated as
363363
candidates for consolidation.
364364
type: integer
365-
default: 10
365+
default: 200
366366
segmentsMin:
367367
description: |
368368
The minimum number of segments that are
369369
evaluated as candidates for consolidation
370370
type: integer
371-
default: 1
371+
default: 50
372372
minScore:
373373
description: |
374374
Filter out consolidation candidates with a score less than this.
@@ -1398,7 +1398,7 @@ paths:
13981398
compaction operations are governed by `consolidationIntervalMsec` and the
13991399
candidates for compaction are selected via `consolidationPolicy`.
14001400
type: integer
1401-
default: 10000
1401+
default: 5000
14021402
consolidationPolicy:
14031403
description: |
14041404
The consolidation policy to apply for selecting which segments should be merged.
@@ -1447,24 +1447,24 @@ paths:
14471447
Defines the value (in bytes) to treat all smaller segments
14481448
as equal for consolidation selection.
14491449
type: integer
1450-
default: 2097152
1450+
default: 25165824
14511451
segmentsBytesMax:
14521452
description: |
14531453
Maximum allowed size of all consolidated segments in bytes.
14541454
type: integer
1455-
default: 5368709120
1455+
default: 8589934592
14561456
segmentsMax:
14571457
description: |
14581458
The maximum number of segments that are evaluated as
14591459
candidates for consolidation.
14601460
type: integer
1461-
default: 10
1461+
default: 200
14621462
segmentsMin:
14631463
description: |
14641464
The minimum number of segments that are
14651465
evaluated as candidates for consolidation
14661466
type: integer
1467-
default: 1
1467+
default: 50
14681468
minScore:
14691469
description: |
14701470
Filter out consolidation candidates with a score less than this.
@@ -1955,24 +1955,24 @@ paths:
19551955
Defines the value (in bytes) to treat all smaller segments
19561956
as equal for consolidation selection.
19571957
type: integer
1958-
default: 2097152
1958+
default: 25165824
19591959
segmentsBytesMax:
19601960
description: |
19611961
Maximum allowed size of all consolidated segments in bytes.
19621962
type: integer
1963-
default: 5368709120
1963+
default: 8589934592
19641964
segmentsMax:
19651965
description: |
19661966
The maximum number of segments that are evaluated as
19671967
candidates for consolidation.
19681968
type: integer
1969-
default: 10
1969+
default: 200
19701970
segmentsMin:
19711971
description: |
19721972
The minimum number of segments that are
19731973
evaluated as candidates for consolidation
19741974
type: integer
1975-
default: 1
1975+
default: 50
19761976
minScore:
19771977
description: |
19781978
Filter out consolidation candidates with a score less than this.

‎site/content/arangodb/3.12/indexes-and-search/arangosearch/arangosearch-views-reference.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -398,7 +398,7 @@ of removing unused segments after release of internal resources.
398398
> subsequent ArangoDB transactions. In-progress ArangoDB transactions
399399
> still continue to return a repeatable-read state.
400400
401-
- **consolidationIntervalMsec** (_optional_; type: `integer`; default: `1000`;
401+
- **consolidationIntervalMsec** (_optional_; type: `integer`; default: `5000`;
402402
to disable use: `0`)
403403

404404
Wait at least this many milliseconds between applying `consolidationPolicy` to
@@ -483,19 +483,19 @@ is used by these writers (in terms of "writers pool") one can use
483483

484484
`consolidationPolicy` properties for `"tier"` type:
485485

486-
- **segmentsMin** (_optional_; type: `integer`; default: `1`)
486+
- **segmentsMin** (_optional_; type: `integer`; default: `50`)
487487

488488
The minimum number of segments that are evaluated as candidates for consolidation.
489489

490-
- **segmentsMax** (_optional_; type: `integer`; default: `10`)
490+
- **segmentsMax** (_optional_; type: `integer`; default: `200`)
491491

492492
The maximum number of segments that are evaluated as candidates for consolidation.
493493

494-
- **segmentsBytesMax** (_optional_; type: `integer`; default: `5368709120`)
494+
- **segmentsBytesMax** (_optional_; type: `integer`; default: `8589934592`)
495495

496496
Maximum allowed size of all consolidated segments in bytes.
497497

498-
- **segmentsBytesFloor** (_optional_; type: `integer`; default: `2097152`)
498+
- **segmentsBytesFloor** (_optional_; type: `integer`; default: `25165824`)
499499

500500
Defines the value (in bytes) to treat all smaller segments as equal for consolidation
501501
selection.

‎site/content/arangodb/3.12/release-notes/version-3.12/api-changes-in-3-12.md‎

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -340,13 +340,29 @@ for related API changes.
340340

341341
#### View API
342342

343+
##### `optimizeTopK` property for `arangosearch` Views
344+
343345
Views of type `arangosearch` accept a new `optimizeTopK` View property for the
344346
ArangoSearch WAND optimization. It is an immutable array of strings, optional,
345347
and defaults to `[]`.
346348

347349
See the [`optimizeTopK` View property](../../indexes-and-search/arangosearch/arangosearch-views-reference.md#view-properties)
348350
for details.
349351

352+
##### Changed consolidation defaults for `arangosearch` Views
353+
354+
<small>Introduced in: v3.12.6</small>
355+
356+
The default values for consolidating `arangosearch` Views have been changed.
357+
By consolidating less often and with more data, less file descriptors are used.
358+
359+
- `consolidationIntervalMsec` increased from `1000` to `5000`
360+
- `consolidationPolicy` (with `type` set to `tier`):
361+
- `segmentsMin` increased from `1` to `50`
362+
- `segmentsMax` increased from `10` to `200`
363+
- `segmentsBytesMax` increased from `5368709120` (5 GiB) to `8589934592` (8 GiB)
364+
- `segmentsBytesFloor` increased from `2097152` (2 MiB) to `25165824` (24 MiB)
365+
350366
#### Document API
351367

352368
The following endpoints accept a new `versionAttribute` query parameter that adds
@@ -463,14 +479,28 @@ add the `withHidden=true` query parameter to the call of the endpoint.
463479
curl "http://localhost:8529/_api/index?collection=myCollection&withHidden=true"
464480
```
465481

466-
#### Vector indexes
482+
##### Vector indexes
467483

468484
<small>Introduced in: v3.12.4</small>
469485

470486
A new `vector` index type has been added.
471487
See [HTTP interface for vector indexes](../../develop/http-api/indexes/vector.md)
472488
for details.
473489

490+
##### Changed consolidation defaults for inverted indexes
491+
492+
<small>Introduced in: v3.12.6</small>
493+
494+
The default values for consolidating inverted indexes have been changed.
495+
By consolidating less often and with more data, less file descriptors are used.
496+
497+
- `consolidationIntervalMsec` increased from `1000` to `5000`
498+
- `consolidationPolicy` (with `type` set to `tier`):
499+
- `segmentsMin` increased from `1` to `50`
500+
- `segmentsMax` increased from `10` to `200`
501+
- `segmentsBytesMax` increased from `5368709120` (5 GiB) to `8589934592` (8 GiB)
502+
- `segmentsBytesFloor` increased from `2097152` (2 MiB) to `25165824` (24 MiB)
503+
474504
#### Optimizer rule descriptions
475505

476506
<small>Introduced in: v3.10.9, v3.11.2</small>

‎site/content/arangodb/3.12/release-notes/version-3.12/incompatible-changes-in-3-12.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -979,6 +979,21 @@ header or an invalid URL, it now sends a response with an error object instead
979979
of closing the connection, e.g. that the URL is corrupt with an HTTP status code
980980
of 400.
981981

982+
## Changed consolidation defaults for inverted indexes and `arangosearch` Views
983+
984+
<small>Introduced in: v3.12.6</small>
985+
986+
The default values for consolidating inverted indexes as well as
987+
`arangosearch` Views have been changed. By consolidating less often and with
988+
more data, less file descriptors are used.
989+
990+
- `consolidationIntervalMsec` increased from `1000` to `5000`
991+
- `consolidationPolicy` (with `type` set to `tier`):
992+
- `segmentsMin` increased from `1` to `50`
993+
- `segmentsMax` increased from `10` to `200`
994+
- `segmentsBytesMax` increased from `5368709120` (5 GiB) to `8589934592` (8 GiB)
995+
- `segmentsBytesFloor` increased from `2097152` (2 MiB) to `25165824` (24 MiB)
996+
982997
## HTTP RESTful API
983998

984999
### JavaScript-based traversal using `/_api/traversal` removed

‎site/content/arangodb/4.0/develop/http-api/indexes/inverted.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -537,7 +537,7 @@ paths:
537537
compaction operations are governed by `consolidationIntervalMsec` and the
538538
candidates for compaction are selected via `consolidationPolicy`.
539539
type: integer
540-
default: 1000
540+
default: 5000
541541
consolidationPolicy:
542542
description: |
543543
The consolidation policy to apply for selecting which segments should be merged.
@@ -570,24 +570,24 @@ paths:
570570
Defines the value (in bytes) to treat all smaller segments as equal for
571571
consolidation selection.
572572
type: integer
573-
default: 2097152
573+
default: 25165824
574574
segmentsBytesMax:
575575
description: |
576576
The maximum allowed size of all consolidated segments in bytes.
577577
type: integer
578-
default: 5368709120
578+
default: 8589934592
579579
segmentsMax:
580580
description: |
581581
The maximum number of segments that are evaluated as candidates for
582582
consolidation.
583583
type: integer
584-
default: 10
584+
default: 200
585585
segmentsMin:
586586
description: |
587587
The minimum number of segments that are evaluated as candidates for
588588
consolidation.
589589
type: integer
590-
default: 1
590+
default: 50
591591
minScore:
592592
description: |
593593
Filter out consolidation candidates with a score less than this.

0 commit comments

Comments
 (0)