2.5.1

Enhancements

• #2484 Limit the number of concurrent connections on .NET Core, similar to Desktop CLR.

• #2451 Small Performance improvements

  Introduce a BufferSize property on JsonNetSerializer with a default size of 1024 to use when writing json to the request stream. Performance tests as part of #1899 indicate this to be a good compromise buffer size for performance throughput and bytes allocated.

• #2495 Add important admonition (end of Simple Automapping section) to the documentation about the use of System.Decimal types as C# POCO properties

Bug fixes

• #2479 Deserialize "value_as_string" for metric aggregations that return it
• #2466 Deserialize attachment title when deserializing an Attachment type from _source or field data
• #2464 Add the StringEnumConverter to StringFielddataFormat to always serialize as string. Introduce coding standard unit test to assert the converter is applied to all NEST enums
• #2487 Fix the virality of &= in the bool query DSL
• #2496 Allow SourceFiltering.Disable to be set to false. This is synonymous with the default, which is to return _source in the response.

Misc

• #2488Tidy up TODOs and make the usage of NotSupportedException consistent across all internal JsonConverter types.
• #2486 Generate asciidoc files as part of the build

View the full list of issues and PRs

5.0.0

GA release of NEST and Elasticsearch.Net 5.0 🎉

A huge thank you to the community who kicked the tires on the 5.0 prereleases that we've been putting out since April and provided us with invaluable feedback 🖖

Check out the blog post on the NEST 5.0 release that highlights some of the main features as well as take a look at the 2.x -> 5.x Breaking Changes for NEST and Breaking Changes for Elasticsearch.Net.

Features

• #2454 Add Support for CreateRequest<T>. This can be used to create a document of a given type in a given index with a given id, with optimistic concurrency control. See https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-index_.html#operation-type for more details
• #2443 Add ScrollAll() helper method

Enhancements

• #2473 Limit the number of concurrent connections on .NET Core, similar to Desktop CLR.

• #2457 Use IndexName and TypeName types for Bulk*Many requests

• #2459 Add type to set index auto expand replica setting

• #2450 Add metadata for Security roles

• #2462 ReindexAll() improvements

  ReindexAll() is now composed over the IObservables from ScrollAll() and BulkAll(), taking advantage of the concurrency models in each.

  Since the rate at which the scrolls produces far exceeds the rate at which we can consume them through bulks ReindexAll() also implements a producer/consumer rate limiter. This is controlled by a backPressureFactor which controls the max amplification factor of running scrolls with the safe guard of:

  searchSize * maxConcurrency * backPressureFactor >= bulkSize

• #2451 Small Performance improvements

  Introduce a BufferSize property on JsonNetSerializer with a default size of 1024 to use when writing json to the request stream. Performance tests as part of #1899 indicate this to be a good compromise buffer size for performance throughput and bytes allocated.

• #2480 Allow target document type to be different from source document type when performing reindex

Bug fixes

• #2479 Deserialize "value_as_string" for metric aggregations that return it
• #2466 Deserialize attachment title when deserializing an Attachment type from _source or field data
• #2464 Add the StringEnumConverter to StringFielddataFormat to always serialize as string. Introduce coding standard unit test to assert the converter is applied to all NEST enums
• #2468 Favour publish_address when sniffing cluster state
• #2467 Fix the virality of &= in the bool query DSL

Misc

• #2456 Tidy up TODOs and #2460 make the usage of NotSupportedException consistent across all internal JsonConverter types.
• #2470 Add test collection filtering when running tests from the command line.

View the full list of issues and PRs

5.0.0-rc4

Features

• #2164 Allow Reindex() helper to reindex into an existing index
• #2436 Add support for Sliced Scrolls

Enhancements

• #2430 Do not instantiate QueryContainerDescriptor<T> unneccessarily
• #2389 Add support for UpdateMany() in the bulk API
• #2418 Faster enum to string resolution
• #2166 Allow DisableDirectStreaming() on a per request basis
• #2336 Allow the key of a bucket aggregation to be read as any type; previously, it was always deserialized to string.

Bug Fixes

• #2431 Add include_in_all onto TypeMapping

• #2420 Fix the way in which HttpClient are created when using the HttpConnection on .NET Core. A HttpClient was created based on a hashcode of RequestData that included a regression bug that caused multiple instances of HttpClient to be created. Now, for RequestData where

  • RequestTimeout
  • HttpCompression
  • ProxyAddress
  • ProxyUsername
  • ProxyPassword
  • DisableAutomaticProxyDetection

  do not change, the same HttpClient instance will be used

• #2428 Add specific type for term vectors in MutliTermVectorsResponse instead of reusing TermVectorsResponse

• #2422 Use ScriptProcessorDescriptor in the fluent API of ingest node

• #2440 Ensure that each part of the async pipeline for Desktop CLR HttpConnection can be cancelled correctly

Deprecations and Removals

• #1958 Allow exceptions late in the pipeline to bubble out as UnexpectedElasticsearchClientException e.g. ResolveException that can happen within the transport when serializing to json.

View the full list of issues and PRs

2.5.0

Features

• #2164 Allow Reindex() helper to reindex into an existing index
• #2413 Implement Watcher APIs 👀

Enhancements

• #2430 Do not instantiate QueryContainerDescriptor<T> unneccessarily
• #2389 Add support for UpdateMany() in the bulk API
• #2418 Faster enum to string resolution
• #2414 Add Min NestedScoreMode

Bug Fixes

• #2431 Add include_in_all onto TypeMapping

• #2420 Fix the way in which HttpClient are created when using the HttpConnection on .NET Core. A HttpClient was created based on a hashcode of RequestData that included a regression bug that caused multiple instances of HttpClient to be created. Now, for RequestData where

  • RequestTimeout
  • HttpCompression
  • ProxyAddress
  • ProxyUsername
  • ProxyPassword
  • DisableAutomaticProxyDetection

  do not change, the same HttpClient instance will be used

• #2142 Set the ApiCall details on each TermVectorsResponse in the collection of responses in a MultiTermVectorsResponse

• #2440 Ensure that each part of the async pipeline for Desktop CLR HttpConnection can be cancelled correctly

View the full list of issues and PRs

5.0.0-rc3

Bug Fixes

• #2412 Fix a number of dictionary serialization issues

A recent change did not handle a number of dictionary serialization cases including

• types that implement IReadOnlyDictionary<TKey,TValue>

  e.g. public class MyReadOnlyDictionary<TKey,TValue> : IReadOnlyDictionary<TKey,TValue>

• types that implement IDictionary<TKey, TValue>

  e.g. public class MyDictionary<TKey,TValue> : IDictionary<TKey,TValue>

• non-generic types that inherit a closed generic dictionary type

  e.g. public class MyDictionary : Dictionary<string,object>

• types that implement IDictionary that are not generic

  e.g. public class MyNonGenericDictionary : IDictionary

5.0.0-rc2

This is the 2nd release candidate compatible with Elasticsearch 5.0. Many features and fixes went into this release, including support for all the Watcher APIs.

We will be finalizing the client ready for a GA release.

Features

• #2347 Support for the Watcher/Alerting API
• #2220 Allow per document index operations on BulkAll
• #2326 Add support for search_after
• #2327 Add support for matrix stats aggregation
• #2334 Add support for Script and DotExpander ingest pipeline processors
• #2335 Infer the index name and type name from T in Suggest<T>()
• #2407 All collections on response types are either IReadOnlyCollection<T> and IReadOnlyDictionary<TKey, TValue>. Additionally, all collections should be initialized.
• #2349 inferred mapping on CLR string properties now sets ignore_above to 256 on the keyword sub field, in line with dynamic mapping in Elasticsearch

Bug Fixes

• #2331 Fix SuffixExpressionVisitor to allow multiple calls of .AppendSuffix()
• #2340 Correct time taken calculation in Audit trail to be non-negative
• #2350 SniffResponse now correctly matches on IPv6 addresses
• #2357 Correct Script serialization on bulk update operation
• #2362 Add max_boundary_scan to highlight fields
• #2344 Add back default value on .Dynamic() mapping
• #2366 404s on Get<T>() and Delete<T>() (and async variants) now return .IsValid as true and deserialize the response, setting .Found.
• #2365 .Score is nullable on Hit<T>

Removal / deprecations

• #2342 Remove Language on UpdateRequest and rename to Lang, and correct script serialization
• #2399 Remove IDictionary from IIsADictionary<TKey,TValue> interface definition. This was confusing and needed only for one purpose in the codebase. This has been refactored to perform better.

View the full list of issues and PRs

2.4.7

Behavioural Breaking Change

• #2367 404s on Get<T>() and Delete<T>() (and async variants) now return .IsValid as true and deserialize the response, setting .Found. This aligns the behaviour of 2.x with 1.x and 5.x. The implications of this are:

1. when issuing a delete, if the delete actually deletes a document or the document to delete does not exist, .IsValid is true
2. when issuing a get where .IsValid is true, .Found should also be checked to determine if the document existed

Features

• #2220 Backport BulkAll from #2162 and allow per document index operations on BulkAll.

BulkAll is designed to parallelize bulk requests to Elasticsearch by iterating a lazily constructed collection of POCO types and partitioning into multiple concurrent bulk requests. It includes simple logic for automatic backoff of requests when documents failed to be indexed due to HTTP 429 response errors and retrying in a configurable number of times in such cases.

Enhancements

• #2371 / #2323 Include the nested identity when performing inner_hits on a nested type
• #2370 Infer the index name and type name from T with Suggest<T>()
• #2309 / #2310 When a ReindexOnServerResponse comes back as 400 status code, deserialize the response so that Failures is populated with more details for why things failed.

Bug Fixes

• #2362 Add max_boundary_scan to highlight fields
• #2373 Fix script serialization when performing bulk updates
• #2357 Correct Script serialization on bulk update operation
• #2331 Fix SuffixExpressionVisitor to allow multiple calls of .AppendSuffix()
• #2319 / #2321 Fix issue where field expressions were incorrectly cached when a dynamic value and a suffix expression were used
• #2308 the Took field was incorrectly mapped as an int value on some responses, instead of long. Since the type cannot be changed (it would break binary compatibility), another field has been introduced, TookAsLong, that will hold the value returned from Elasticsearch; Took will also contain the value and for values greater than int, Took will be int.MaxValue. Took is marked with Obsolete attribute to highlight this to client consumers and Took is mapped as long in NEST 5.x. thanks @maeserichar 👍

Deprecations and Removals

• #2346 Mark properties removed in 5.0 with ObsoleteAttribute
• #2297 Mark MissingQuery with ObsoleteAttribute as removed in 5.0
• #2293 Mark IndexName on property mappings with ObsoleteAttribute as removed in 5.0

View the full list of issues and PRs

5.0.0-rc1

This is the Elasticsearch 5.0.0-rc1 compatible release.

Features

#2322 Mappings: Added support for half_float and scaled_float numeric datatypes
#2337 Updated datatype properties to align with ES 5.0 across the board
#2320 Added ability to supply local metadata to mapping properties TY @ejsmith
#2292 Allow implicit conversion from string[] to Types and Indices

Bug Fixes

#2209 Several improvements in some Query DSL edge cases
#2339 SniffingConnectionPool: Fixed response parsing of _nodes when sniffing cluster state
#2319 Fixed caching of field expressions that contained both variables and constants
#2310 ReindexOnServer: Correctly handle 400 responses
#2308 Fixed deserialization of responses that contain took, which was incorrectly typed to int. TY @maeserichar !
#2295 XML doc fix. TY @niemyjski !
#2242 #2269 Enhancements to serialization of DateTime
#2195 Query on PhraseSuggestCollate is now correctly typed to ITemplateQuery

Removal / deprecations

#2297 Removed deprecated MissingQuery
#2293 Removed deprecated IndexName

View the full list of issues and PRs

5.0.0-beta1

This is the Elasticsearch 5.0.0-beta1 compatible release.

Features

#2233 Support for get task API
#2228 Support for multi search template API
#2228 Support for shrink index API
#2226 Support for rollover index API
#2246 Support for disable/enable user APIs of the X-Pack security plugin
#2241 Support for field_masking_span query (TY @yevhen!)
#2198 HttpClientHandler can now be modified in a custom HttpConnection when running on .NET Core. This makes it possible to validate HTTPS certs by setting ServerCertificateValidationCallback.
#2197 & #2247 Introduced an improved approach to ModifyJsonSerializerSettings for overriding Json.Net serialization settings using ISerializerFactory instead (TY @jonyadamit !)

Bug Fixes & Enhancements

#2274 Fix NRE when deserializing a mutli-bucket aggregate with an empty buckets object
#2261 Add missing routing option to FieldLookup
#2209 & #2209 Several improvements in some edgecase query DSL scenarios
#2221 Fixed issue with using a sniffing connection pool where a thread could potentially become blocked resulting in a timeout exception (TY @lostllama)
#2219 Add missing conflicts option to reindex API
#2215 Add missing term vector filters
#2215 add missing metadata on IGetResponse<T> and IMultiHit<T>
#2169 Move analyze query string parameters to request body
#2194 Removed deprecated Alias on GetAliasRequest (use Name instead) and added back the GetAliasesPointingToIndex() helper method.
#2193 Add missing collate_match option to geo_shape queries (TY @HarveyWilliams !)
#2187 Add relation to geo_shape queries and contains to geo_shape relations
#2178 Add ability to specify a custom similarity using both the fluent and object iniitalizer syntax

2.4.6

Features

• #2112 Support ReindexRethrottle when running against Elasticsearch 2.4.0, to allow throttling of longer running tasks such as Reindex and UpdateByQuery APIs.
• #2231 Support field_masking_span queries. Mapped to SpanFieldMaskingQuery to align with names of other span queries.

Enhancements

• #2279 Support overloaded UpdateByQuery() fluent methods that infer Indices and Types from generic parameter type T
• #2247 Enable conveniently overriding JsonSerializerSettings without the need to implement a custom serializer. Thanks for the PR @jonyadamit 👍

Bug Fixes

• #2261 FieldLookup now includes a Routing property. ty @seanwm
• #2178 Fix support for custom similarities using CustomSimilarity object or .Custom() fluent method on SimilaritiesDescriptor
• #2170 Resolve to the smallest whole time unit when working with DateMath as it does not support fractional values. For example, TimeSpan.FromHours(25) will serialize to 25h as opposed to round to 1d or fractional value 1.04d. ty @emmekappa
• #2209 Improve boolean combinatorial logic when dealing with bool queries with many clauses
• #2168 Mark .Highlights on SearchResponse<T> as obsolete, to be removed in 5.x. It's a leaky abstraction when searching over multiple types that contain the same document id for different types - this will throw an exception. Highlights can be enumerated using

foreach (var hit in searchResponse.HitsMetaData.Hits)
{
    foreach (var highlight in hit.Highlights)
    {
        // do something with highlightHit
        var highlightHit = highlight.Value;
    }
}

• #2125 Allow use of custom highlighters (expert level option). HighlightField now contains a CustomType string property that can be used to specify a custom highlighter. Core highlighters can still be specified using Type property.
• #2119 Allow use of per field custom similarity (expert level option). ElasticsearchPropertyAttributeBase and associated mapping methods allow the use of a custom similiarity through a CustomSimilarity string property (on attributes and mapping objects) and through the .Similairy(string) fluent mapping method. Core similarities can still be specified.
• #2274 Fix NullReferenceException when deserializing a multi-bucket aggregation that contains an empty bucket.
• #2263 Do not use IsoDateTimeConverter to handle DateTimeOffset. Take a look at #2268 for how DateTime and DateTimeOffset are serialized within NEST.