Change Log
==========

All notable changes to this project will be documented in this file.


## v2.5.0


### Event reporting

- [#3d3f3a5](https://github.com/Mellanox/sharp/commit/3d3f3a5f567ae4cec98ce10060a670ec0eebbadc): SHARP Python API: SMX messages updates
- [#6321f51](https://github.com/Mellanox/sharp/commit/6321f51c2ad3ff36575ab74b785ad693f2e6599c): SHARP Python API: Add support for new events
- [#77c580b](https://github.com/Mellanox/sharp/commit/77c580b58301085f0184e623e183129b7a4a7697): AM: Add reservation ID to Job Error events
- [#2595e1c](https://github.com/Mellanox/sharp/commit/2595e1c0046a41b7af2984602d78d15e6a07733d): AM: Send job error event on sharpd disconnection
- [#550fc7c](https://github.com/Mellanox/sharp/commit/550fc7cb015fad3b84d570584a2944e9d867736e): AM: Add new events generation
- [#497c949](https://github.com/Mellanox/sharp/commit/497c94961932546c877f9911aed536964b3b66e7): AM: Add support for privileged connection
- [#37b66ca](https://github.com/Mellanox/sharp/commit/37b66ca23f0293f3753531496f703719b0394475): SHARP API: Add support for privileged connection
- [#57e6fbb](https://github.com/Mellanox/sharp/commit/57e6fbbb88cbdc9320098d733337802bf484c91b): SMX: Add support for privileged connection


### Connection keep alive

- [#17e6605](https://github.com/Mellanox/sharp/commit/17e6605ce19f97cb241c02942afaa92928582b3c): AM: Support keep-alive in AM
- [#bb1f1c4](https://github.com/Mellanox/sharp/commit/bb1f1c4584f89de0d4a43b83cc0ef2266d6d0c3c): SMX: Support keep-alive with UCX
- [#0e2a29d](https://github.com/Mellanox/sharp/commit/0e2a29d712757f261bcc8829cb27844e0ab16efb): SMX: Support keep-alive with socket
- [#d580fa7](https://github.com/Mellanox/sharp/commit/d580fa7ef49f2fe1ac2e27113f6a06b9e31151e8): SHARPD: Add keep-alive configuration
- [#df28829](https://github.com/Mellanox/sharp/commit/df288293dd3473c8c5fbb52d049ae17f14043b0e): CI: Jenkins test keep-alive


### NDR support

- [#0d6241b](https://github.com/Mellanox/sharp/commit/0d6241b3e2a814203aac9278473fcd8fba8f4373): AM: Support MTU 4K
- [#049f7f1](https://github.com/Mellanox/sharp/commit/049f7f1b733e4bb2d5d5bc1f7edc634557fa89dd): AM: Support new data types in switches
- [#0050c13](https://github.com/Mellanox/sharp/commit/0050c139604f0be91ec7f645f7883c411876eb10): AM: SAT/LLT QP Bookkeeping
- [#bacbbbe](https://github.com/Mellanox/sharp/commit/bacbbbebd37e4e9cafa4d47181f27f6c7c9379a1): AM: Add support for fp19 and bfloat19 calculations
- [#5b597e1](https://github.com/Mellanox/sharp/commit/5b597e1223f687aae41a0b48e8349b167fd2207c): AM: Aggregation trees prioritization according to aggregation rate
- [#3b8583d](https://github.com/Mellanox/sharp/commit/3b8583df4019ef57d0c8c26c4aa0bdb0a2433a05): AM: Build max trees supported by network devices.
- [#e95fa41](https://github.com/Mellanox/sharp/commit/e95fa4190750fd7a2893508b9b251524731b9c7e): SMX: Add fp19 and bfloat19 related types
- [#e5c748b](https://github.com/Mellanox/sharp/commit/e5c748ba82a4be003425aff624d7fd0b0f271316): SHARPD: Support flags fp19 and bfloat19
- [#8e33aae](https://github.com/Mellanox/sharp/commit/8e33aaeb692779da2c3d39482efa7a8cb9022ce8): sharp_api/coll Add support for fp19 and bfloat19 operations


### Changed

- [#6c34b8f](https://github.com/Mellanox/sharp/commit/6c34b8f430bbd19c69bc91f04bbf17eb976c383a): sharp_coll: Option to retry END JOB on failure
- [#dcec7a4](https://github.com/Mellanox/sharp/commit/dcec7a44044accbcfbd242dec68b8deef69d15a6): sharp_coll: Add MTU, hop_limit in debug print
- [#f5455d0](https://github.com/Mellanox/sharp/commit/f5455d01c46ab91579ab15783d16c9be693b9499): sharp_coll: Add option for sharp coll log file
- [#bfed1f7](https://github.com/Mellanox/sharp/commit/bfed1f789172eb461707d8596e8e29fabe093b23): sharp_coll: lock SAT tree is comm init only if batch size is Infinity
- [#8fe1ac7](https://github.com/Mellanox/sharp/commit/8fe1ac73283b0e90866d89a264de1bd104c0f0ab): sharp_coll: Option to use DEVX
- [#82b1270](https://github.com/Mellanox/sharp/commit/82b1270ce363e0a6ec725ffbd3066a53c1f0a534): sharp_coll: Print tree ID in lock failure error log
- [#c86c785](https://github.com/Mellanox/sharp/commit/c86c785b664fcc36fc20843361f1f4b82f85b362): AM: Send only one job end in sharp_error trap
- [#83fe753](https://github.com/Mellanox/sharp/commit/83fe753eeab54dc36f36bceae1b1a7d89a78275b): AM: Mark AN as invalid if clean job tree failed
- [#6da4275](https://github.com/Mellanox/sharp/commit/6da4275bab47139d250915b07083855657697787): AM: Improve logging of fabric update messages
- [#be8bc3c](https://github.com/Mellanox/sharp/commit/be8bc3ce35f574c89961586f92f9df9a25d94410): AM: QPError Trap - check if Aggregation Node
- [#0d1ed60](https://github.com/Mellanox/sharp/commit/0d1ed60ecbfe9cc9773a4e9653e87cf2b6bc2ee3): AM: Set 'tid' in SMX message header in JobData response.
- [#966e63b](https://github.com/Mellanox/sharp/commit/966e63bf3b085e5d003f49ad80b9f58238749e65): AM: Use sharp_signal_handler for signals handling
- [#5edd0e5](https://github.com/Mellanox/sharp/commit/5edd0e5fb43d77ede60e311b7d27faac1d674faf): AM: Set default max_tree_radix to maximal value (#2555)
- [#12ff6d2](https://github.com/Mellanox/sharp/commit/12ff6d21e4fc780ebd3237dc2b25350f8b1df85e): SMX: Move declaration between header files
- [#b53bbac](https://github.com/Mellanox/sharp/commit/b53bbacc4569764be4a9dac057177e945ac4a4f0): SMX: Change log verbosity
- [#57342c9](https://github.com/Mellanox/sharp/commit/57342c9426d61fc91b0fa4edefb3530372f81eac): SHARPD: Fix releasing socket pair on sub-thread
- [#39d5e6e](https://github.com/Mellanox/sharp/commit/39d5e6e9473a7c7c4d966166629f4b868b192410): SHARPD: Change SR MAD retry timeout from 50msec to mad_timeout 2sec
- [#b411bcb](https://github.com/Mellanox/sharp/commit/b411bcbc5d363f974408877b5272f25e91ee6b0e): SHARPD: Tune group join max retry and timeout
- [#2977b08](https://github.com/Mellanox/sharp/commit/2977b08896d1dd837ddeca13936032d693f72d4a): SHARPD: Support caching service records
- [#b0ba2b5](https://github.com/Mellanox/sharp/commit/b0ba2b5cac306b2d97668ff08cbcf44884f76c7a): SHARPD: Add subnet prefix to management port info
- [#a00874a](https://github.com/Mellanox/sharp/commit/a00874acf4ed667490820de3977785823b42fb83): SHARPD: Change SMX callback signature
- [#913b496](https://github.com/Mellanox/sharp/commit/913b496c92f73cccc8bc53ee2ff467bef289c909): AM/SHARPD: Allow writing all log levels to syslog
- [#5a9dcf2](https://github.com/Mellanox/sharp/commit/5a9dcf2c098bfca8b04a014248662c47e9e13e60): AM/SHARPD: Increase default SR request timeout
- [#9106e0b](https://github.com/Mellanox/sharp/commit/9106e0b1e06b7e4a678ea868ac7c7be56f659111): SHARP API: Update smx_config in swig files
- [#4ae581e](https://github.com/Mellanox/sharp/commit/4ae581e735482f1907dbcbc98a548aa821bc3a7c): SHARP API: Clear "tid to callback" map in SharpSmx destructor
- [#65e9d1c](https://github.com/Mellanox/sharp/commit/65e9d1c645222a21889e3bf0292d41af8dfa9354): SHARP API: cosmetic changes
- [#7a05126](https://github.com/Mellanox/sharp/commit/7a0512694645a42674aba4cef2389d4a53df2c86): Jenkins: Update fabric update test
- [#b58c56b](https://github.com/Mellanox/sharp/commit/b58c56b0b84784f0b352a9e30b7af8ef4d27536c): Tests fix: Change MTU enum


### Fixed

- [#a42df18](https://github.com/Mellanox/sharp/commit/a42df183e1e79bf6e0a4a0a0834ca9c53137aa14): AM: Fix topology API for more than 2 level trees
- [#d3207c5](https://github.com/Mellanox/sharp/commit/d3207c50292d26e7314310e75e65b49ad12fa977): AM: Fix Host to switch connection changes
- [#760cecc](https://github.com/Mellanox/sharp/commit/760ceccb042eb1ed135e1bd669a91050ff0f70de): SHARP: Fix truncating PID file
- [#69b31c7](https://github.com/Mellanox/sharp/commit/69b31c70b348691dae5c973ff9d65fbc19ca74a4): SMX: Fix disconnection control msg send
- [#f6259c9](https://github.com/Mellanox/sharp/commit/f6259c9b369af23ac669632733587340a66563d1): SMX: Close unix socket on exit
- [#60b8551](https://github.com/Mellanox/sharp/commit/60b8551bb31e6ddf68b13b33e28f6453d6dc985b): AM: Pass log level as uint8 instead of int
- [#0522e8e](https://github.com/Mellanox/sharp/commit/0522e8ea1e3522e3c5d497fc96f66ce70b56f0c3): AM: Build fix
- [#582637b](https://github.com/Mellanox/sharp/commit/582637b5bd31f1fff8576d77190eb2e91601cb14): SYSTEMD: Fix path for PID file
- [#469aabc](https://github.com/Mellanox/sharp/commit/469aabccd63c8484b887d292b188e1e8a6d88048): Tests fix: Add SMDB file and AN2AN section
- [#d53aad3](https://github.com/Mellanox/sharp/commit/d53aad39b0d581cda34f1ecebd944c09a30ce56e): AM: Bug fix - Missing 'AggRate' parameter in PathUpdate
- [#7c81468](https://github.com/Mellanox/sharp/commit/7c81468e374ff012597ecc1cc989a4b8f6228b25): AM: Bug fix - Events messages typos


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/a42df183e1e79bf6e0a4a0a0834ca9c53137aa14)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.6...a42df183e1e79bf6e0a4a0a0834ca9c53137aa14)


## v2.4.6


### Changed

- [#21d2914](https://github.com/Mellanox/sharp/commit/21d2914a8629aa83c08d9a3c66bf8c500620830b): AM: Add parameter to explicitly allow using Topology API
- [#645b487](https://github.com/Mellanox/sharp/commit/645b4879bb798b66d46ff231cc5676c5538253bf): AM: Allow querying topology API without reservations
- [#79aefb3](https://github.com/Mellanox/sharp/commit/79aefb3279d69d107f72d82dfe23f571b2de33a9): AM: Add 'info' prints in fabric graph
- [#e1ace1c](https://github.com/Mellanox/sharp/commit/e1ace1c802a58e709adb198e7195a494f3e83f21): AM: Support parsing new virtualization file after startup (#2567)
- [#d25b366](https://github.com/Mellanox/sharp/commit/d25b36684373492f3872bf86f59cdd8d4cddfd20): AM: Improve logging of traps
- [#d2a4066](https://github.com/Mellanox/sharp/commit/d2a4066cf99f916d932bde9a25bd76afc2834c65): AM: Improve Job Error flow
- [#d4467c5](https://github.com/Mellanox/sharp/commit/d4467c5b2e978e757cae4bf0b76cf7bd8e8aa497): SMX: Add smx_ctor link constructor
- [#0cb35a7](https://github.com/Mellanox/sharp/commit/0cb35a74879a5ad0f6ae201e83860a6b5d156bf2): SMX: Return error if UCX configured but built UCX
- [#a4947ea](https://github.com/Mellanox/sharp/commit/a4947eab446156309cc8407d2ef1f81b2ab62e4d): configure.ac: Update configure, ucx.m4
- [#7e54e92](https://github.com/Mellanox/sharp/commit/7e54e9211b74a453b1362f9d26449c2ae936227e): sharp_coll: add sharp_job_id in INFO log level


### Fixed

- [#1948367](https://github.com/Mellanox/sharp/commit/19483671af01fd1a8ffc2a264be4055de7f19b4b): AM: Bug fix uninitialized value in AM Key
- [#fd7713c](https://github.com/Mellanox/sharp/commit/fd7713cfd71f91647c336f9f0228edc1d2744ac7): AM: Bug fix missing resource cleanup on AN recovery flow
- [#b329857](https://github.com/Mellanox/sharp/commit/b3298572ed9f414885f425b1163c086d6ca99bf0): SMX: Fix deadlock on stress
- [#721d03f](https://github.com/Mellanox/sharp/commit/721d03f36e2fea547fd815c078d382d744785301): SMX: Fix in invalid read in str2ep for UCX address
- [#f9ede7f](https://github.com/Mellanox/sharp/commit/f9ede7fa0253926580192a8e31032128474d4238): SMX: Fix null-terminated string in ep2str for UCX addrress
- [#bfb71e9](https://github.com/Mellanox/sharp/commit/bfb71e99b920d666a5058f9f86130f926f9bb3f0): sharpd: Fix uninitialized memory
- [#015ba32](https://github.com/Mellanox/sharp/commit/015ba32b6b026e9a50e24440abd6a7b305acaa8e): sharp_coll: Fix race condition with SHARP_COLL_SAT_LOCK_BATCH_SIZE=1


### Changes in parameters

| Parameter                     |Component|  Status     |  Description                          |
|-------------------------------|---------|-------------|---------------------------------------|
| enable_topology_api           | AM      |  New        | Enable/Disable topology API feature   |


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/21d2914a8629aa83c08d9a3c66bf8c500620830b)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.5...21d2914a8629aa83c08d9a3c66bf8c500620830b)


## v2.4.5


### Added

- [#8c3d51b](https://github.com/Mellanox/sharp/commit/8c3d51bfd7d0c0de5e71b5ffdbf610f4d5244572): sharp_am: Add support for 'exclusive_lock' feature
- [#9624408](https://github.com/Mellanox/sharp/commit/9624408e6c4c0c1660f65b691b956dd1cdfa54bd): sharp_api/coll Add support for 'exclusive_lock' feature
- [#cb1ffc9](https://github.com/Mellanox/sharp/commit/cb1ffc96a99d87e355e4aa6d188c4b8662667372): sharpd: Add 'exclusive_lock' attribute to sharp_job/sharpd_create_job_req
- [#8ff0e31](https://github.com/Mellanox/sharp/commit/8ff0e315e3a068cd1f0331a5f81dafb04afee8f3): smx: Add sharp_exclusive_lock enum to sharp_begin_job
- [#7c26807](https://github.com/Mellanox/sharp/commit/7c2680729b54de1f562e7e1cb1210c18d742e32e): option_manager: Add 'enable_exclusive_lock' attribute


### Changed

- [#7c3c223](https://github.com/Mellanox/sharp/commit/7c3c2237d64933c3f238f08633e209f69a96146f): Add switch description to trees dump file
- [#54193ae](https://github.com/Mellanox/sharp/commit/54193aecf7815d5a221aed914e852779c48cd875): Sharp_am: Create persistent directory during CommandManager Initialization
- [#a65b6bc](https://github.com/Mellanox/sharp/commit/a65b6bc24a281d895dc3fcb0546a1b95bc08fc0f): sharp_coll: Separate blocking and non-blocking progress


### Fixed

- [#3448337](https://github.com/Mellanox/sharp/commit/3448337608681bf5e277411884c234b09ce2f923): sharp_pkt.h: Fix ppc64 be compilation
- [#2542](https://github.com/Mellanox/sharp/pull/2542): sharp_coll: Fix build on ppc64 BE
- [#fd08434](https://github.com/Mellanox/sharp/commit/fd08434ce1000f37a438d21553fe764e31337905): CommandManager: Wrong enum type used as Signal
- [#9f88936](https://github.com/Mellanox/sharp/commit/9f889365c0d18f42976a001b7bef0fe6e9f54727): sharp_coll: Fix pending non-blocking SAT request return code
- [#75285f5](https://github.com/Mellanox/sharp/commit/75285f569273cc36b80c3d608094f65f6205e9f9): Cosmetic changes: Fix includes (#2512)


### Changes in parameters

| Parameter                     |Component|  Status     |  Description                          |
|-------------------------------|---------|-------------|---------------------------------------|
| enable_exclusive_lock         | AM      |  New        | Enable/Disable exclusive lock feature |


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/7c3c2237d64933c3f238f08633e209f69a96146f)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.4...7c3c2237d64933c3f238f08633e209f69a96146f)


## v2.4.4


### Fixed

- [#da128bc](https://github.com/Mellanox/sharp/commit/da128bc1b654d14a9129eca69fc93168779ead6d): sharp_am: Topology API bug fix (#2531)


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/da128bc1b654d14a9129eca69fc93168779ead6d)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.3...da128bc1b654d14a9129eca69fc93168779ead6d)



## v2.4.3


### Fixed

- [#2526](https://github.com/Mellanox/sharp/pull/2526): sharp_py_smx: Stop timer fixes
- [#2521](https://github.com/Mellanox/sharp/pull/2521): fabric_graph.cpp - Bug: Race condition may cause AM to crush


### Changed

- [#2524](https://github.com/Mellanox/sharp/pull/2524): sharp_am: Change minhop table type from map to vector
- [#2525](https://github.com/Mellanox/sharp/pull/2525): AM: Expose device_configuration_file parameter
- [#2518](https://github.com/Mellanox/sharp/pull/2518): AM: Do not load reservations on startup by default
- [#bcfdb74](https://github.com/Mellanox/sharp/commit/bcfdb743a13c43295a43cf464eec809030d207ee): SharpApi: Add Enum SharpEventType


### Changes in parameters

|Parameter                      |Component|  Status     |  Description                          |
|-------------------------------|---------|-------------|---------------------------------------|
| device_configuration_file     | AM      |  Exposed    |                                       |


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/329f86258281d96939931b9168f29291ed2b925b)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.2...329f86258281d96939931b9168f29291ed2b925b)


## v2.4.2


### Changed

- [#243a14f](https://github.com/Mellanox/sharp/commit/243a14f05b1027578967c2049492bcd7f10afa58): coll/coll_stats.c: fix compilation issues
- [#2516](https://github.com/Mellanox/sharp/pull/2516): CHANGELOG.md: Add v2.4.1 content


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/243a14f05b1027578967c2049492bcd7f10afa58)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.1...243a14f05b1027578967c2049492bcd7f10afa58)


## v2.4.1

### Changed

- [#efcfe48](https://github.com/Mellanox/sharp/commit/efcfe488051a5132c81db2aeeb755862f57db65a): amkey_manager: Change severity of log message


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/efcfe488051a5132c81db2aeeb755862f57db65a)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.4.0...efcfe488051a5132c81db2aeeb755862f57db65a)


## v2.4.0


### Added

- [#df80b52](https://github.com/Mellanox/sharp/commit/df80b52ba57011f4ab3aa90eefec3a517a96a2bc): Tests: Gtests for EVENTS request/reply msgs
- [#e13896d](https://github.com/Mellanox/sharp/commit/e13896d105eeef0402f44533f3955229d290154d): sharp_api.py - Events integration
- [#232e1a7](https://github.com/Mellanox/sharp/commit/232e1a7a231322184024567264ac2c6d04397f7a): sharp_am: Add EventManager initialization
- [#1e1bd5f](https://github.com/Mellanox/sharp/commit/1e1bd5f82a70b8ac7eb5e274b8946761651d46e5): EventManager: New Class
- [#36ace4b](https://github.com/Mellanox/sharp/commit/36ace4bb3fc0788796046c9d6feb56c4b5984fdc): SMX: Add events related msgs
- [#73581bc](https://github.com/Mellanox/sharp/commit/73581bcb08dcfabc934901ceff6e27ac0d189d9b): option_manager: Add configuration attributes for EventsManager
- [#cc34f81](https://github.com/Mellanox/sharp/commit/cc34f81f4c78c7c3d55ba8b60390c73abadd049f): SHARPD: Send END_JOB message to AM on shutdown
- [#c0bc9d9](https://github.com/Mellanox/sharp/commit/c0bc9d94f728efaf5b60ee245b1aa60a73aa260d): sharpd: Support duplicate external job IDs
- [#9fac835](https://github.com/Mellanox/sharp/commit/9fac835327a195401202aa232f396e4a774b580b): sharp_am: Support duplicate external job IDs
- [#2472](https://github.com/Mellanox/sharp/pull/2472): AMKey pending mode
- [#7056d7e](https://github.com/Mellanox/sharp/commit/7056d7eac015e85821cb34cfe820885e9bc2f969): AM: Identify ANs according to SpecialPortType field from SMDB


### Changed

- [#bfa31c2](https://github.com/Mellanox/sharp/commit/bfa31c254280a4f84907473f2b3ed33b81c10bed): .gitmodules: Change ibis branch to sharp-2.4
- [#0ac9dab](https://github.com/Mellanox/sharp/commit/0ac9dab1771698344a7698df1ba3e5196b5cd0c1): sharp_am: Overwrite existed reservation file
- [#ce394f9](https://github.com/Mellanox/sharp/commit/ce394f95151ccaeadd93de00096e5e2b23546af1): smx_py: Update smx types in py_smx_types.i file
- [#1c1f270](https://github.com/Mellanox/sharp/commit/1c1f2705fadb1c579a53f3ad06021d186c432763): return proper error code from allreduce_nb progress func
- [#d85f946](https://github.com/Mellanox/sharp/commit/d85f9468aff8d4301bf647e6a291bbc861bb8e40): configure.ac: Increment libsharp version
- [#a9be9a9](https://github.com/Mellanox/sharp/commit/a9be9a91dd0106bc5f731555e5eebadbd43d348e): Add documentation for device configuration file
- [#7f6cd39](https://github.com/Mellanox/sharp/commit/7f6cd396a4a467ef1f3acac9c6aaae76d71da767): sharp_am: Update tests for local_ack_timeout and timeout_retry_limit
- [#06428c8](https://github.com/Mellanox/sharp/commit/06428c8da203534c06e13b045d4bcad6e5a217f6): sharp_am: Update defaults for local_ack_timeout and timeout_retry_limit
- [#0065645](https://github.com/Mellanox/sharp/commit/0065645799d1c7d0bf3c1b19a4e1fd3b6c857c6a): sharp_coll: Add debug infra for SAT lock race
- [#2864f62](https://github.com/Mellanox/sharp/commit/2864f62d0eb7c7878243affc57738b069a7c7264): sharp_am: set default ib_max_mads_on_wire to 4096
- [#8ccb6ac](https://github.com/Mellanox/sharp/commit/8ccb6ac40c1d36b338797b83f0da10a2accd88e5): Gtests for new field reservation_id
- [#75e57b3](https://github.com/Mellanox/sharp/commit/75e57b3076a6365c0e4fc71a80c2690eb299b5a8): smx: Add new field reservation_id to smx messages
- [#e41cd15](https://github.com/Mellanox/sharp/commit/e41cd1549d3f616ef4bea1da5cb3ef0c4b9e9ea9): fabric: Remove irrelevant log message
- [#900d130](https://github.com/Mellanox/sharp/commit/900d1300c73b3ab82fc170a9009bba80d68b201b): sharp_api.py: Cosmetic fix
- [#1678946](https://github.com/Mellanox/sharp/commit/167894618ba07ea8db8fbe6d65528e2339c8a229): sharp_api.py: Validate 'int' input bit length
- [#82e6537](https://github.com/Mellanox/sharp/commit/82e6537165fe5970178373672b0468d7cf9bfa34): sharp_smx.py: Add new status value to reservation status
- [#b26a0fd](https://github.com/Mellanox/sharp/commit/b26a0fd7a98ddcff6a5ba163217336558748d12f): libsharp_coll: Remove variable declaration from header
- [#3a39b0d](https://github.com/Mellanox/sharp/commit/3a39b0daaa8800f3f171ecf727ec69880e2a5f4f): sharpd: Remove variable declaration from header
- [#e801971](https://github.com/Mellanox/sharp/commit/e8019711f81f6a5bd077bc82ba37d81037a51695): sharp_coll:  option set explict jobid with sharp_coll_test/sharp_mpi_test


### Fixed

- [#3ab3eb9](https://github.com/Mellanox/sharp/commit/3ab3eb913fbcf497219bcd885f7781266d917903): sharp_am: End job if no reserved GUIDs left in job
- [#f4b26da](https://github.com/Mellanox/sharp/commit/f4b26da53e9c7e1f77d9d94b6f19dfb513a59186): sharp_am: Fix bug in  AggNode reset malfunction
- [#07df126](https://github.com/Mellanox/sharp/commit/07df126854a0f3cda540ab6bb29d253f7c0101b6): AM: Fix leak in FabricGraph.m_node_by_rank_
- [#dde706a](https://github.com/Mellanox/sharp/commit/dde706afe10169045a31ca5383292f0048f934a4): SHARPD: Fix legacy coverity issues
- [#83fe562](https://github.com/Mellanox/sharp/commit/83fe562856f9106c21ffb067a9876d1c09b02597): SHARPD: Fix checking of signalfd return value
- [#a9ad752](https://github.com/Mellanox/sharp/commit/a9ad7528331a8dc113487a706f79b6270a068c61): sharp_am: Fix QP local ACK timeout parameter description
- [#0c79bd0](https://github.com/Mellanox/sharp/commit/0c79bd07d22e8d13a8581bb6810792a6300e998e): Jenkins test fixes for reservation_id field
- [#ccba160](https://github.com/Mellanox/sharp/commit/ccba16085027a57f8bcea8a6a786c420acc89411): sharp_timer.py: Check log callback before usage
- [#8a136e3](https://github.com/Mellanox/sharp/commit/8a136e30b0f22e925057c946f6508e98c82adf55): smx.py: Bug fixes
- [#543d8ec](https://github.com/Mellanox/sharp/commit/543d8ece49ea16509931b2ff20e757c639d9c067): libsharp_coll: Initialize sharp_trees memory
- [#2485](https://github.com/Mellanox/sharp/pull/2485): fabric_graph: Remove redundant code


### Changes in parameters

|Parameter                        |Component|  Status     |  Description                          |
|---------------------------------|---------|-------------|---------------------------------------|
| ib_max_mads_on_wire             | AM      |  Changed    | Changed default from 100 to 44096     |
| ib_qpc_local_ack_timeout        | AM      |  Changed    | Changed default from 0x1F to 0x12     |
| ib_sat_qpc_local_ack_timeout    | AM      |  Changed    | Changed default from 0x1F to 0x12     |
| ib_qpc_timeout_retry_limit      | AM      |  Changed    | Changed default from 7 to 6           |
| ib_sat_qpc_timeout_retry_limit  | AM      |  Changed    | Changed default form 7 to 6           |

- [HEAD](https://github.com/Mellanox/sharp/commit/bfa31c254280a4f84907473f2b3ed33b81c10bed)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.3.3...bfa31c254280a4f84907473f2b3ed33b81c10bed)


## v2.3.2

### Topology API

- [#6c02a13](https://github.com/Mellanox/sharp/commit/6c02a13553b00722da94062f07c98ee79f4bf255): sharp_cmd: Utility to send topology request msg
- [#7e67755](https://github.com/Mellanox/sharp/commit/7e67755db97c667803188f6379b41fb9634be49f): sharpd: Support send cmd messages
- [#6144e91](https://github.com/Mellanox/sharp/commit/6144e91a7410552e2a69fb0f040e05fa11006965): sharpd: Allow sending TID in smx messages header to AM.
- [#d82feb4](https://github.com/Mellanox/sharp/commit/d82feb40ef42bad532f7b59848f9cfe007f01e79): sharp_opt_parser: Support unnamed parameters


### Fixed

- [#46877f3](https://github.com/Mellanox/sharp/commit/46877f3bfbf245666b897a2c50b3a969e7ab62e7): sharp_opt_parser: Fix show usage printouts


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/6c02a13553b00722da94062f07c98ee79f4bf255)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.3.1...6c02a13553b00722da94062f07c98ee79f4bf255)


## v2.3.1

### Cloud HPC

- [#a9161ed](https://github.com/Mellanox/sharp/commit/a9161ed20c5fdb67b88195a656436801a449c3b9): SHARPD: Send MADs to ANs with job pkey
- [#f030b56](https://github.com/Mellanox/sharp/commit/f030b56bfdbe0a1dbe65a8262a1731f9efaf3361): AM: Fix disabling multicast for jobs with non-default pkey
- [#a706347](https://github.com/Mellanox/sharp/commit/a706347eca2a12e928acfd6ce945874e3c0b4e4b): SMX: Support setting pkey for UCX worker
- [#0172d86](https://github.com/Mellanox/sharp/commit/0172d862596bd2c3c9e991a4f8e0900a2db4abcc): AM: Reset SMX configuration struct before usage
- [#a86ac4b](https://github.com/Mellanox/sharp/commit/a86ac4ba1b17ec13aaf46e442c443915c86ec4d3): sharpd: Send MADs to Aggregation nodes with pkey index 0
- [#e4a7efe](https://github.com/Mellanox/sharp/commit/e4a7efe23b0fb77553898587b58f0f97375c2c14): sharp_am: Check duplicate GUIDs in create reservation
- [#46a9f76](https://github.com/Mellanox/sharp/commit/46a9f76ff97bee7d6eeb042aa6679a700a299db7): sharp_am: Update reservation query reply
- [#de00fc7](https://github.com/Mellanox/sharp/commit/de00fc70ace5126fd300ebc7e466ad19d1e16566): sharp_am: Cosmetic changes in reservation manager
- [#2452](https://github.com/Mellanox/sharp/pull/2452): src/am/resource_manager.cpp: Take Pkey from job_resource
- [#049f267](https://github.com/Mellanox/sharp/commit/049f267b8836b69f19c89f25d093d7897a4e21f8): AM: Delete reservation without terminating active jobs


### Topology API

- [#6168671](https://github.com/Mellanox/sharp/commit/61686718f396123f735c67598f0969b9bae54078): sharp_am: Bug fix in Topology request
- [#2b191cd](https://github.com/Mellanox/sharp/commit/2b191cd4fa7564aa6308e1ac77e24a36602c997f): sharp_am: Add status field to topology smx msg
- [#ea833c7](https://github.com/Mellanox/sharp/commit/ea833c7ffa3a24c6e4689e225be77716cee82b71): sharp_am: Topology API
- [#e280ad7](https://github.com/Mellanox/sharp/commit/e280ad7e8ca6d55cce3dcbbe3586aca3db4b377e): Tests: Gtest for topology request/reply msgs
- [#45e9715](https://github.com/Mellanox/sharp/commit/45e971589c4a3e365caa5a3fead1efa2dd4af612): smx: Add new smx msgs. Topology request/reply
- [#e3e8ca4](https://github.com/Mellanox/sharp/commit/e3e8ca45c3748e45360d5bf6dea308d8eba6eb8c): AM: Calculate minimal hops between leaf switches


### Added

- [#09fd525](https://github.com/Mellanox/sharp/commit/09fd52576688afdf3aea160d1da2c5ca3bc859fe): sharp_coll_test: Add datatype option for coll test
- [#2363](https://github.com/Mellanox/sharp/pull/2363): Support build armcc
- [#2454](https://github.com/Mellanox/sharp/pull/2454): Support Service Key


### Fixed

- [#0320ecd](https://github.com/Mellanox/sharp/commit/0320ecd7a86a13af48aa528532fea6eb88fb17f3): smx_gen.py: smx txt mode - Coverity fix
- [#c589729](https://github.com/Mellanox/sharp/commit/c589729907a24b3cada66f53968739f9fce3d108): smx_gen.py: Fix smx msg release function
- [#541e021](https://github.com/Mellanox/sharp/commit/541e0217378eb7b7f8b380a9803fac9bf8563306): Tests: Fix AM tests in exp files
- [#9d8d0f4](https://github.com/Mellanox/sharp/commit/9d8d0f4c87f1dfa19139bd5f952fe90ffdb05e95): sharpd: Support big messages from libsharp
- [#6b07c97](https://github.com/Mellanox/sharp/commit/6b07c97c3746c0865d54f8d9279f4aab971b3d4a): Setting Ibis branch to master
- [#2470](https://github.com/Mellanox/sharp/pull/2470): Fix hpc-arm build
- [#2464](https://github.com/Mellanox/sharp/pull/2464): 2148448: SHARP: AM Key support
- [#38cf1cd](https://github.com/Mellanox/sharp/commit/38cf1cd53b9636e193b392223b9c363213bd684f): AM: Fix Configure AN port resources after AN info
- [#7e2827a](https://github.com/Mellanox/sharp/commit/7e2827a39904eecc15348b0bbbaf78f06005369e): smx_proc: Fix smx get local address
- [#578ac61](https://github.com/Mellanox/sharp/commit/578ac617d3a95afee426dfb5a984e4e96d2f3536): smx_proc: Add\Remove fds on connect\disconnect request
- [#86c5283](https://github.com/Mellanox/sharp/commit/86c52831beb18e9dbe592d9d169ea7c781605198): SMX: Ignore unknown fields in SMX text mode
- [#ef5f677](https://github.com/Mellanox/sharp/commit/ef5f677b325383da6d93c6a9f4f03a0226c4782d): sharp_am,sharpd: Fix empty string in smx interface
- [#ae06bfc](https://github.com/Mellanox/sharp/commit/ae06bfcec1b79569ff71cbb3f0f0937348dcdac7): sharp_coll: Fix bug in MINLOC/MAXLOC op with non-blocking API.


### Changed

- [#26c2725](https://github.com/Mellanox/sharp/commit/26c272597ceba8a5a36bfd48fdb1e2f95d2520ec): sharp_api.py: Change reporting status in API
- [#d6381df](https://github.com/Mellanox/sharp/commit/d6381dff975c8a07fd55344b9dfaa236e2984c72): sharp_am_smx_address: Support new SR format
- [#c28bc66](https://github.com/Mellanox/sharp/commit/c28bc666a670296cfdbba4a9ff67eeac848e520b): SMX: Set UCX IB address type to ib_global in order to use GRH
- [#417f3ca](https://github.com/Mellanox/sharp/commit/417f3cae95fa5d1ec09bba9e45946dbc9746ea06): Fix coverity issues
- [#757c034](https://github.com/Mellanox/sharp/commit/757c034bd5f12d631a6ece839ef376d7eae51522): AM: expose smx_protocol parameter
- [#d3722e7](https://github.com/Mellanox/sharp/commit/d3722e752ea714f8b60227d868eaaf99514692a2): sharp_smx.py: Cosmetic changes
- [#1af3d75](https://github.com/Mellanox/sharp/commit/1af3d750c02bfb217e9e041b5db968a17ecc55e7): sharp_smx.py: Change ID generate function
- [#0284c89](https://github.com/Mellanox/sharp/commit/0284c8965a226aa11d08a028aefb2d85f658a636): AM: Expose smx_enabled_protocols parameter
- [#50c219c](https://github.com/Mellanox/sharp/commit/50c219cb28dade319e2c200d8da843013bd62067): sharp_am, sharpd: smx_enable_protocol range [1-7]
- [#ca0e679](https://github.com/Mellanox/sharp/commit/ca0e67959ae1a7afce88589b10fd978ea2a2b3a5): remove google protobuf


### Changes in parameters

|Parameter                      |Component|  Status     |  Description                          |
|-------------------------------|---------|-------------|---------------------------------------|
| smx_protocol                  | AM      |  New        | Control default SMX protocol          |
| smx_enabled_protocols         | AM      |  New        | Control enabled SMX protocols         |
| ib_service_key                | AM      |  New        | Service key for AM service record     |
| smx_sock_interface            | AM      |  Changed    | Fix invalid default value             |
| smx_ucx_interface             | AM      |  Changed    | Fix invalid default value             |
| smx_sock_interface            | SD      |  Changed    | Fix invalid default value             |
| smx_ucx_interface             | SD      |  Changed    | Fix invalid default value             |


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/61686718f396123f735c67598f0969b9bae54078)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.2.0...61686718f396123f735c67598f0969b9bae54078)


## v2.2.0

### Cloud HPC

- [#460c644](https://github.com/Mellanox/sharp/commit/460c6443f468957114b68d6836be3f754b68845d): sharp_am: Support reservations
- [#2422](https://github.com/Mellanox/sharp/pull/2422): sharpd: Send MADs to ANs and SM with management PKEY
- [#2418](https://github.com/Mellanox/sharp/pull/2418): Support Pkey index lookup
- [#2421](https://github.com/Mellanox/sharp/pull/2421): src/am/job_manager.cpp: Disable multicast for non default pkeys jobs
- [#2415](https://github.com/Mellanox/sharp/pull/2415): Support PKey from begin job
- [#2397](https://github.com/Mellanox/sharp/pull/2397): Send MADs with GRH
- [#2402](https://github.com/Mellanox/sharp/pull/2402): GRH support

### sharp_coll

- [#41a2773](https://github.com/Mellanox/sharp/commit/41a2773645eb8d3fe9273de3979b21d32a44277e): sharp_coll: check for ibv_alloc_null_mr
- [#33a2072](https://github.com/Mellanox/sharp/commit/33a207218c20521688cbbdaf20ba0b09ca7771a3): sharp_coll_test:  Implement BCAST validation and perf tests
- [#f02839e](https://github.com/Mellanox/sharp/commit/f02839e312f87843c029c0b053545c4443a4b069): sharp_coll: Implement BCAST over Allreduce
- [#2548970](https://github.com/Mellanox/sharp/commit/2548970bd9182139fc0d7f90e063acce5e6c90d0): configure/cuda: add lib64/stubs to LDFLAGS
- [#44b2731](https://github.com/Mellanox/sharp/commit/44b2731d3278ffbd584f1ddbd5779767201973cc): sharp_coll: Tuning OSTs per group
- [#d1cad43](https://github.com/Mellanox/sharp/commit/d1cad439683a423d9b4feccfe7925243d82ce88b): sharp_coll: add external progress to sharp mpi benchmark
- [#c9bd240](https://github.com/Mellanox/sharp/commit/c9bd240d1e64cceda2b95248b26624691ae97513): sharp_coll: SHARP SAT lock retry timeout option
- [#0754a7c](https://github.com/Mellanox/sharp/commit/0754a7ca59939338a35cd97204195eb09c89d73c): sharp_coll: explicit unlock operation
- [#de44aa0](https://github.com/Mellanox/sharp/commit/de44aa0514380ab4bf7a56dc7bb1ffd812f493c8): sharp_coll: option to try SAT unlock in case of failed to unlock

### Python API

- [#cdbf409](https://github.com/Mellanox/sharp/commit/cdbf4093d6e646d45667ca8f78f6c36c7322f08a): sharp_api.py: Add description to SharpStatus
- [#3faeabb](https://github.com/Mellanox/sharp/commit/3faeabbb66ffde4f2c7ea069eb6623e9be0a54b9): sharp_api.py: Python API to send smx messages

### SMX

- [#f5060d0](https://github.com/Mellanox/sharp/commit/f5060d0ba20d30e5b7a1fd0c65ff3bb6f38860ac): configure.ac: Use CHECK_UCX macro from ucx.m4
- [#93cdfdf](https://github.com/Mellanox/sharp/commit/93cdfdf8f3145e8242d060f5b5221d3431eea9e0): Add ucx.m4 to config/m4
- [#c9e1a09](https://github.com/Mellanox/sharp/commit/c9e1a0947499110f0dfaa3c785849c9cbf10ca7c): src/smx/Makefile.am: use UCX_LIBADD instead of UCX_LIBS
- [#52a6f9d](https://github.com/Mellanox/sharp/commit/52a6f9d959d070f171896fd945101a5f042c362f): SMX: Add new smx messages and gtests
- [#4cac088](https://github.com/Mellanox/sharp/commit/4cac088690dc9f741342ef2be17938410194d00e): SMX_gen.py: smx bug fixes
- [#65eeed9](https://github.com/Mellanox/sharp/commit/65eeed97207810bc8330259ea714a67d6cd56c2e): SMX: Support unix socket
- [#2424](https://github.com/Mellanox/sharp/pull/2424): Set UCX transport to DC
- [#2373](https://github.com/Mellanox/sharp/pull/2373): SMX: Choose connection type according to ep type
- [#0e91592](https://github.com/Mellanox/sharp/commit/0e91592f72b283f59eaf419c99645fedb82d4da5): SMX: Add UCX temporary workaround. (#2404)
- [#2372](https://github.com/Mellanox/sharp/pull/2372): Support UCX in SHARP SMX

### Fixed

- [#1605244](https://github.com/Mellanox/sharp/commit/160524483196e714a60bb7d31adb36fffe7a58a2): sharp_am: Fix switch rank change restored
- [#2b9f790](https://github.com/Mellanox/sharp/commit/2b9f7901cad5b53fddf8ecb8f198e4290950cf5d): sharp_client: Fix sharp_disconnect
- [#3cc0380](https://github.com/Mellanox/sharp/commit/3cc0380d6dbecc4ab0aa3e2d83ffca7f2dd5d379): sharp_coll_test: Fix hugethp madvise return check

### Changes in parameters

|Parameter                      |Component|  Status     |  Description                                            |
|-------------------------------|---------|-------------|---------------------------------------------------------|
| enable_job_pkey_on_tree       | AM      | New         | Set job PKEY on aggregation tree allocated for the job  |
| reservation_mode              | AM      | New         | Enable reservations in AM                               |
| smx_ucx_interface             | AM      | New         | Network interface to be used by SMX for UCX connections |
| smx_ucx_interface             | SHARPD  | New         | Network interface to be used by SMX for UCX connections |


- [HEAD](https://github.com/Mellanox/sharp/commit/160524483196e714a60bb7d31adb36fffe7a58a2)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.1.3...160524483196e714a60bb7d31adb36fffe7a58a2)


## v2.1.4

### Fixed

- [#79f0692](https://github.com/Mellanox/sharp/commit/79f069228600119b95feafdeb12f1b2915a5f1b8): sharp_am: fix handling reset switch connected to AM (#2408)

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/79f069228600119b95feafdeb12f1b2915a5f1b8)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.1.3...79f069228600119b95feafdeb12f1b2915a5f1b8)

## v2.1.3

### Fixed

- [#bf5539b](https://github.com/Mellanox/sharp/commit/bf5539bfa8846c7f79f6f6c8e4d513af84521d49): sharp_coll: Fix race when GDRCOPY is disabled
- [#d552be3](https://github.com/Mellanox/sharp/commit/d552be325b78a244149760b4b7587dfbf643a797): change wording
- [#042d629](https://github.com/Mellanox/sharp/commit/042d62914d01759bde9b3e19850916b5c80d22f6): sharp_coll: Workaround for SAT auto unlock latency in lock_per_op mode
- [#0e0dc5c](https://github.com/Mellanox/sharp/commit/0e0dc5c810e42c1bcaa4c025f80315e073bf95c7): sharpd: Fix race condition when removing job
- [#9cd1866](https://github.com/Mellanox/sharp/commit/9cd1866fbde0cea25bfb8faa433da43638ae5b70): sharpd.c: Add lock to sharp_build_job_list_reply_message

### Changed

- [#2399](https://github.com/Mellanox/sharp/pull/2399): sharp_am: Set SAT QP default SL to 1
- [#a8e4bb3](https://github.com/Mellanox/sharp/commit/a8e4bb371964e381598f436b1af5d744aeccd46f): Enable building sharp_am separately (#2396)
- [#f6b18de](https://github.com/Mellanox/sharp/commit/f6b18de300a06f3816227f74b5561f1abef9dda3): TEST: sharp mpi test enhancements

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/76d1f1a8b29f701af70d030bd98a878be4db493e)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.1.2...76d1f1a8b29f701af70d030bd98a878be4db493e)


## v2.1.2

### Changed

- [#3bcbc86](https://github.com/Mellanox/sharp/commit/3bcbc86938a90c173cde10e2fcba9e7e8cace2ca): sharp_coll: Enable packet based credits by default
- [#5a19a3d](https://github.com/Mellanox/sharp/commit/5a19a3d9fee14fa10a9634e7062d604130d3f5fb): sharp_coll_test: Add reduce test
- [#8c8d8e0](https://github.com/Mellanox/sharp/commit/8c8d8e005bd8653e220d1c37633de4738d7b5dd6): sharp_coll: implement Reduce operation on HBA tree
- [#8cc22e6](https://github.com/Mellanox/sharp/commit/8cc22e6aa5595b61db07f97fa696661924c24b52): sharp_coll: Set default SAT threshold to 16k.
- [#2381](https://github.com/Mellanox/sharp/pull/2381): CHANGELOG.md: Add v2.1.1 content

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/3bcbc86938a90c173cde10e2fcba9e7e8cace2ca)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.1.1...3bcbc86938a90c173cde10e2fcba9e7e8cace2ca)


## v2.1.1

### Fixed

- [#2376](https://github.com/Mellanox/sharp/pull/2376): sharp_am: Bug fix. QP error after switch restart during SAT job. 
- [#2378](https://github.com/Mellanox/sharp/pull/2378): sharp_am: Fix configuring port resources every update
- [#7cabb8a](https://github.com/Mellanox/sharp/commit/7cabb8a5da6e5463f9cca8a09fa4f748b2a4f23c): jquota: Fix sharp_init_session return check
- [#7d5d4f6](https://github.com/Mellanox/sharp/commit/7d5d4f697a602b3e4d6ca79c62ef441ec7a1ca68): sharp_coll: Fix resource atomicity for multi-thread case

### Changed

- [#6490f3d](https://github.com/Mellanox/sharp/commit/6490f3d4a117673e63fd97c9294825a4c4e0c3e2): sharp_coll: Disable Scatter to CQE with upstream on SAT qp to support GPUDirectRDMA
- [#0ab5f42](https://github.com/Mellanox/sharp/commit/0ab5f42b128701e3400fa160a8c41c34fdadbee4): sharp_coll: Add support of PCI RELAXED ORDERING
- [#70a2735](https://github.com/Mellanox/sharp/commit/70a2735a84b34e1df51a4b4fab757fc1f1bed316): sharp_coll: Packet-Based credits support with upstream mlx5dv
- [#23dac4f](https://github.com/Mellanox/sharp/commit/23dac4f841d78c04de781079aa4c9770a164d56e): configure: Add upstream mlx5dv support
- [#7645611](https://github.com/Mellanox/sharp/commit/76456116cb84ee18bf1f717e8e88ae4467437082): Remove unused configurations
- [#002f6c6](https://github.com/Mellanox/sharp/commit/002f6c67460a08e2677dd9438040b6600bdfc1ff): Applied the review remarks.
- [#b5b60a2](https://github.com/Mellanox/sharp/commit/b5b60a2a17455a83bf90555bb074bf1c910b0b09): Applied the review remarks.
- [#2bec8d3](https://github.com/Mellanox/sharp/commit/2bec8d399170afb6959b8f1a2f4187b6fc1d4bea): Applied the review remarks.
- [#a45df4f](https://github.com/Mellanox/sharp/commit/a45df4f40654b302638e4ca50819f9800a811385): Applied the review remarks.
- [#928d7d6](https://github.com/Mellanox/sharp/commit/928d7d64156c89dc5703d8b982f9e47e8871f65b): Applied the review remarks.
- [#ea44005](https://github.com/Mellanox/sharp/commit/ea44005e90a75e56e975c26e949cdac06c7b9a1b): Enabled SHARP-NCCL CI


## v2.1.0

### Main features

#### Setting topology type based on routing engine in smdb file

- [#f251dcfa6c](https://github.com/Mellanox/sharp/commit/f251dcfa6c3cd3a16d9ffb8e43917968b5eff00f): AM: set topology type according to routing engine in smdb file

### Fixed

- [#6fc80c1672](https://github.com/Mellanox/sharp/commit/6fc80c1672363dd6de348c1244c70e9ad6eb5a10): AM: fix job id allocation (#2351)
- [#2348](https://github.com/Mellanox/sharp/pull/2348): sharp_am: fix writing MADs to files
- [#2346](https://github.com/Mellanox/sharp/pull/2346): sharp_coll: Fix possible stack overflow
- [#2341](https://github.com/Mellanox/sharp/pull/2341): sharp_coll:  Fix lock retry with timeout and lock statistics 
- [#2331](https://github.com/Mellanox/sharp/pull/2331): sharp_coll_test: Fix Bw reporting in Gbs
- [#2342](https://github.com/Mellanox/sharp/pull/2342): Fix partial read in libhsarp
- [#3e59c1f996](https://github.com/Mellanox/sharp/commit/3e59c1f9965bf5b42c41f40d6cb771746bf4595a): sharp_coll: Fix external user progress in blocking api
- [#cede932f09](https://github.com/Mellanox/sharp/commit/cede932f097a6b4300631db8b32c9a360f20f105): AM: build fix, remove unused variable (#2352)
- [#13762e4f3f](https://github.com/Mellanox/sharp/commit/13762e4f3f6e4f7f8a8986827866b7e599048bef): sharp_coll: Force packed based e2e credits on switch SAT QP
- [#2330](https://github.com/Mellanox/sharp/pull/2330): sharp_coll: Avoid recursive calling of non-blocking SAT request

### Changed

- [#d5c14f8b4b](https://github.com/Mellanox/sharp/commit/d5c14f8b4b78492e2632c901ac6b3df7f67e8dee): sharp_col: Set default SAT lock retry count to 1
- [#2350](https://github.com/Mellanox/sharp/pull/2350): sharp_coll: Lazy group resource allocation
- [#2356](https://github.com/Mellanox/sharp/pull/2356): sharp_coll: Tuning OST payload size

### Added

- [#2349](https://github.com/Mellanox/sharp/pull/2349): sharp_am: Support additional PortCreditResourcesAllocation fields
- [#2347](https://github.com/Mellanox/sharp/pull/2347): sharp_am: Support AnPortCreditResourcesAllocation
- [#2345](https://github.com/Mellanox/sharp/pull/2345): sharp_coll: Set status when last event finished
- [#a398f4abe1](https://github.com/Mellanox/sharp/commit/a398f4abe1f1f4c36240e851cafa99c1bf8377eb): Support SharpError trap
- [#cc2ab7a3a7](https://github.com/Mellanox/sharp/commit/cc2ab7a3a7dfed67450abb1bfe021ae5f84d0374): Support trap QP error in SAT. Destroy edge in switch. Send job error at the end of qp error handle.
- [#896c8be3c3](https://github.com/Mellanox/sharp/commit/896c8be3c32c79070ab1d5f1099fa1214784deae): SAT job resource cleanup

### Details

- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.0.0...v2.1.0-rc1)

## v2.0.0

## Main features

### sharp_coll streaming aggregation

- [#2321](https://github.com/Mellanox/sharp/pull/2321): sharp_coll: GDRCOPY-2.0 API support
- [#87a6e75](https://github.com/Mellanox/sharp/commit/87a6e75654d6a5a7e4ae1bf9f976bde38d230ce3): sharp_coll: Disable packet based credits by default (#2306)
- [#2293](https://github.com/Mellanox/sharp/pull/2293): sharp_coll: Option to Lock SAT on group init time
- [#2279](https://github.com/Mellanox/sharp/pull/2279): sharp_coll: Options to retry SAT lock
- [#2248](https://github.com/Mellanox/sharp/pull/2248): sharp_coll: Option to set SAT lock batch size
- [#2256](https://github.com/Mellanox/sharp/pull/2256): sharp_coll: Separate cuda and gdrcopy wrappers to avoid dependency
- [#2252](https://github.com/Mellanox/sharp/pull/2252): sharp_coll: Add Makefile and README for sharp_coll_test distribution
- [#2255](https://github.com/Mellanox/sharp/pull/2255): sharp_coll: Set feature list in sharp_coll_caps_query
- [#2247](https://github.com/Mellanox/sharp/pull/2247): sharp_coll: De-allocate handle memory in sharp_coll_req_wait
- [#2241](https://github.com/Mellanox/sharp/pull/2241): sharp_coll: enable cuda support in try mode.
- [#2238](https://github.com/Mellanox/sharp/pull/2238): sharp_coll: use cuda API through wrapper lib
- [#2234](https://github.com/Mellanox/sharp/pull/2234): sharp_coll: Add hugepage allocation for test buffers
- [#2233](https://github.com/Mellanox/sharp/pull/2233): sharp_coll: registration cache for gdrcopy 
- [#2232](https://github.com/Mellanox/sharp/pull/2232): sharp_coll: link gdrcopy lib only to libsharp_coll
- [#2230](https://github.com/Mellanox/sharp/pull/2230): sharp_coll: Add gdr_copy support
- [#2229](https://github.com/Mellanox/sharp/pull/2229): sharp_coll: Add support for IOV data buffers
- [#2204](https://github.com/Mellanox/sharp/pull/2204): sharp_coll: Option to disable packet-based credits explicitly 
- [#87da3b7](https://github.com/Mellanox/sharp/commit/87da3b70c776e26daf61d175696b778a97fc4a29): sharp_coll: hold HBA lock for  group life time (#2197)

### AM / SHARPD streaming aggregation

- [#504f35c](https://github.com/Mellanox/sharp/commit/504f35cfe4e3d131e3a9a1b351f91693d3801c7a): sharpd.c: Allow force switch credits (#2305)
- [#2285](https://github.com/Mellanox/sharp/pull/2285): AM: Construct multiple trees
- [#2253](https://github.com/Mellanox/sharp/pull/2253): Support SAT load per tree ID
- [#2241086](https://github.com/Mellanox/sharp/commit/2241086b98cc6452efdfa90ba4fa8825ce12f65f): sharpd: disable trimming for SAT (#2236)
- [#2215](https://github.com/Mellanox/sharp/pull/2215): job_manager.cpp: Generalize multiple ports single tree case
- [#2f1515b](https://github.com/Mellanox/sharp/commit/2f1515b9a50bee40f54d7012e271eb4a93564158): sharpd.c: Refactor add_tree_conns (#2190)
- [#2183](https://github.com/Mellanox/sharp/pull/2183): Jquota: Support SAT
- [#2b82965](https://github.com/Mellanox/sharp/commit/2b82965397f14b4c667ab25076b61d99c344c9a0): sharp_am: Enable packet based credits on AN-AN streaming aggregation QPs (#2224)
- [#2206](https://github.com/Mellanox/sharp/pull/2206): Build at least two trees
- [#2202](https://github.com/Mellanox/sharp/pull/2202): Handle an to an rerouting
- [#fe15aae](https://github.com/Mellanox/sharp/commit/fe15aaef077125fcea433d81564009076f9f5b69): sharpd: Disable packet_based_credit_req_en for SAT (#2210)

### SMX

- [#2303](https://github.com/Mellanox/sharp/pull/2303): smx2 msg pack alignment
- [#2284](https://github.com/Mellanox/sharp/pull/2284): SMX: Support backward and forward compatibility to SMX
- [#2f3517a](https://github.com/Mellanox/sharp/commit/2f3517ab6b95dbc1e6d2e085dd8a45ec2924d76a): Gtest: persistent_job_info smx tests
- [#d28e8d4](https://github.com/Mellanox/sharp/commit/d28e8d469e4248012a7b13cddb5526f4fde68b92): protobuf: build cpp version only (#2203)

### Added

- [#adb1d4e](https://github.com/Mellanox/sharp/commit/adb1d4efb636bbb26afb4faf380ee2eeb80a0f2a): SHARP: new service record format (#2316)
- [#2228](https://github.com/Mellanox/sharp/pull/2228): sharp_am: Support dynamically calculating QPs per port
- [#2310](https://github.com/Mellanox/sharp/pull/2310): sharp_coll_test:  COLL test updates
- [#e53947a](https://github.com/Mellanox/sharp/commit/e53947a71baa6119e9dbdb0953f78d1ca46fdba3): sharp: support compilation on linux rdma-core
- [#55d1306](https://github.com/Mellanox/sharp/commit/55d13063e72e1bb40417529fc0b58bd3c56bde44): sharpd: Remove dependency on libibmad
- [#5836c87](https://github.com/Mellanox/sharp/commit/5836c870b8cbf792ee054906f59db1348f8eb995): AM: persistent job smx serialization
- [#2295](https://github.com/Mellanox/sharp/pull/2295): Jenkins: Build NCCL-RDMA-SHARP plug-in
- [#2287](https://github.com/Mellanox/sharp/pull/2287): Reproducibility per job
- [#4cff932](https://github.com/Mellanox/sharp/commit/4cff93299b2466578ad4625c3f75203761a2f23d): sharpd.c: implement retry policy for busy group_join
- [#6331274](https://github.com/Mellanox/sharp/commit/633127458801169eee0921d38e8171dfa0b83367): Jenkins: create valgrind log at the end. (#2290)
- [#fa56237](https://github.com/Mellanox/sharp/commit/fa56237d876b95d61e7d1c0f8253c27ae1055c38): Jenkins: Increase lockfile time (#2289)
- [#2267](https://github.com/Mellanox/sharp/pull/2267): Support socket-based activation on installation
- [#2266](https://github.com/Mellanox/sharp/pull/2266): AM pending mode
- [#2276](https://github.com/Mellanox/sharp/pull/2276): Support multiple client sessions per process
- [#2265](https://github.com/Mellanox/sharp/pull/2265): BUILD: RPM distribution for CUDA support.
- [#2239](https://github.com/Mellanox/sharp/pull/2239): API: remove unused variables from public API
- [#3d50ead](https://github.com/Mellanox/sharp/commit/3d50eadd36b9e730096132d0d7adaad9b69bf1c4): SHARPD: Query service records with sa_key 0
- [#ab47df6](https://github.com/Mellanox/sharp/commit/ab47df69e201798cf8f1490fef960431e924cc23): AM: Support configurable sa_key for service records
- [#84c1ad9](https://github.com/Mellanox/sharp/commit/84c1ad95fe6ef72e6b2f5dfc18023ec4488b55d3): Store MGIDs in network order (#2235)
- [#b6c9b8a](https://github.com/Mellanox/sharp/commit/b6c9b8acba7e90ebf971fb59517cb2cf415455e8): hcoll_build.sh: Add sharp_dir to LD_LIBRARY_PATH (#2226)
- [#2222](https://github.com/Mellanox/sharp/pull/2222): sr.h: Add space between literal and identifiers in logging macros
- [#e5e5a24](https://github.com/Mellanox/sharp/commit/e5e5a243460ac80c3927e91612ead55de38dc99b): am tests: Update fabric_update test for Updated QPs recovery
- [#2278](https://github.com/Mellanox/sharp/pull/2278): sharp_coll: use random job ID for coll test
- [#2243](https://github.com/Mellanox/sharp/pull/2243): sharp_coll: Use external oob_ctx instead NULL for sharp coll init OOB

### Fixed

- [#3b47ae1](https://github.com/Mellanox/sharp/commit/3b47ae1104996524ba63b708489471c30ec6b031): AM: Check for errors after reading the job file (#2323)
- [#2322](https://github.com/Mellanox/sharp/pull/2322): sharp_coll: Check NULL mem handle for SAT
- [#2319](https://github.com/Mellanox/sharp/pull/2319): sharp_coll: Decrease packet-based credits disabled log  from info to debug
- [#2318](https://github.com/Mellanox/sharp/pull/2318): sharp_coll; Fix port_idx when SAT and multi-rail is enabled
- [#2311](https://github.com/Mellanox/sharp/pull/2311): sharp_coll: Fix debug build on upstream
- [#7482718](https://github.com/Mellanox/sharp/commit/748271853a733759e70dc2885661971af3d61302): sharpd: Fix reading MC proxy join response (#2312)
- [#39df3e8](https://github.com/Mellanox/sharp/commit/39df3e829a1c501bbb5e8d2f6fc8d93575dd42fb): AM: stop fd_listener before sr_cleanup
- [#d2f7647](https://github.com/Mellanox/sharp/commit/d2f7647b120b41cf7ac8f96b4cc4167b52d5995a): Fix AM tests race condition (#2281)
- [#2262](https://github.com/Mellanox/sharp/pull/2262): COLL_TEST: Fix Makefile
- [#f7fde96](https://github.com/Mellanox/sharp/commit/f7fde9658f8316fec597172b8ff9347b464d9019): AM: Fix IS_SAT_TREE_ID macro (#2254)
- [#2245](https://github.com/Mellanox/sharp/pull/2245): sharp_coll: Fix issue with Zero length non blocking allreduce request
- [#2242](https://github.com/Mellanox/sharp/pull/2242): sharp_coll: Fix hang with multi threaded support
- [#2244](https://github.com/Mellanox/sharp/pull/2244): sharp_coll: Fix invalid write in the cuda lib path
- [#5415a8a](https://github.com/Mellanox/sharp/commit/5415a8a1a7a6e0a0e2126a8041b00cc514c75515): contrib/jenkins_tests/coll_test.sh: Fix LD_LIBRARY_PATH in coll_test.sh (#2237)
- [#2214](https://github.com/Mellanox/sharp/pull/2214): Fix fedora28 issues
- [#2217](https://github.com/Mellanox/sharp/pull/2217): am tests: Fix tests to support Build idintical fat trees
- [#96af953](https://github.com/Mellanox/sharp/commit/96af953218859f37fd355ac1ec3e59b35022cd93): Typo in debug prints
- [#23f8165](https://github.com/Mellanox/sharp/commit/23f8165240732858ad4ba865f3f4c5a649a87573): Sharpd error flow fd leak
- [#02c211c](https://github.com/Mellanox/sharp/commit/02c211c0d7f5f101f351008078ab6f73a9ce1d1a): sharp_am: Fix race condition on shutdown (#2292)
- [#25fa1a5](https://github.com/Mellanox/sharp/commit/25fa1a5748425c927d9ffb3fc56509e366e836d5): smx: changed smx log verbosity. from "info" to "trace". (#2297)
- [#1a62617](https://github.com/Mellanox/sharp/commit/1a6261782a91315492724713621870070266c842): AM: fix warning on struct zero initialization (#2302)
- [#fb66e97](https://github.com/Mellanox/sharp/commit/fb66e9739b4d1a878a71b567c94ccedc730cbb6d): Coverity fix: check smx pack 'rc' and if job content pointer is null.
- [#2291](https://github.com/Mellanox/sharp/pull/2291): sharp_coll_test:  coll test fixes
- [#04db818](https://github.com/Mellanox/sharp/commit/04db818086ab368baf87c9d4b704ffd21d5a7864): Memory leak fixes (#2272)
- [#2219](https://github.com/Mellanox/sharp/pull/2219): V2.0 fix sl
- [#3219ea5](https://github.com/Mellanox/sharp/commit/3219ea5d8b0fb5fd638385dd2d990207a4e89ddf): fix gtest compilation on ubuntu18.10 (#2205)

### Changed

- [HEAD](https://github.com/Mellanox/sharp/commit/3b47ae1104996524ba63b708489471c30ec6b031)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v2.0.0-alpha...3b47ae1104996524ba63b708489471c30ec6b031)


## v2.0.0-alpha

## Main features

### HDR support and streaming aggregation

- [#42a4c8c](https://github.com/Mellanox/sharp/commit/42a4c8c69d5841c29e5be0140ba01a4c89cdea80): sharp_am: Update QPs recovery to support SAT
- [#2194](https://github.com/Mellanox/sharp/pull/2194): sharp_coll: SAT non-blocking implementation
- [#9aed91b](https://github.com/Mellanox/sharp/commit/9aed91bf59374067c4cc91fcbe1062102ce6f59c): libsharp_coll: Calculate group_id for tuple header in libsharp_coll (#2193)
- [#2189](https://github.com/Mellanox/sharp/pull/2189): sharp_coll: set SAT Qp path_mtu to IBV_MTU_2048
- [#4f5d5e7](https://github.com/Mellanox/sharp/commit/4f5d5e72216d1980aa4ecad63214db3278bf509b): SHARPv2: Add flags to sharp_tree_connect_info (#2187)
- [#2186](https://github.com/Mellanox/sharp/pull/2186): sharp_coll: do not consider osts and payload_per_ost of SAT group in
- [#2180](https://github.com/Mellanox/sharp/pull/2180): sharp_coll: SAT QP with packet based credit mode
- [#d22745d](https://github.com/Mellanox/sharp/commit/d22745d87245a0b4445d0c4cd83945baf789e370): jquota: Set max OST payload to 1K (#2181)
- [#2179](https://github.com/Mellanox/sharp/pull/2179): sharp_coll: Add SAT support
- [#2172](https://github.com/Mellanox/sharp/pull/2172): SHARPD: Support SAT groups allocation
- [#2163](https://github.com/Mellanox/sharp/pull/2163): Allocate SAT
- [#2176](https://github.com/Mellanox/sharp/pull/2176): Add peer_tree_id to sharp_tree_info
- [#2175](https://github.com/Mellanox/sharp/pull/2175): shrp_coll/client: set cap.max_payload attribute to 1024
- [#2173](https://github.com/Mellanox/sharp/pull/2173): sharp_coll: remove option SHARP_COLL_MAX_PAYLOAD_SIZE
- [#2149](https://github.com/Mellanox/sharp/pull/2149): SHARP_COLL: SAT API and flags
- [#2150](https://github.com/Mellanox/sharp/pull/2150): SHARP_COLL: Set sharp job id in tuple header
- [#41ac7df](https://github.com/Mellanox/sharp/commit/41ac7dff31e95d297d546cb0d91e21b0967cf9c8): sharp_am: Update job_data sat_qpc_options
- [#64d8516](https://github.com/Mellanox/sharp/commit/64d85162c980d3365fb73acc2409e6d53808deb2): sharp_am: Add build and update SAT trees test
- [#1dc8529](https://github.com/Mellanox/sharp/commit/1dc8529c5fabf0c371a6f74d04ea8bda6e98a35a): sharp_am: Send control path version to Fabric on Start
- [#50bc3bd](https://github.com/Mellanox/sharp/commit/50bc3bdf6eb84e2ddf843d6c6ddfc730ed521baf): sharp_am: Add enable_sat configuration parameter
- [#3d2b703](https://github.com/Mellanox/sharp/commit/3d2b703b5d9cf903b9c42101ff2b91de01366b8e): sharp_am: Configure SAT QPs
- [#38cbbd2](https://github.com/Mellanox/sharp/commit/38cbbd2d93cd5d426ab4b293864caeff22362676): sharp_am: Configure SAT on fabric
- [#6a51568](https://github.com/Mellanox/sharp/commit/6a5156867259c9c2dc580a68859c184764b0da57): sharp_am: Add sw_hops count to AggPath
- [#a8f9346](https://github.com/Mellanox/sharp/commit/a8f93461d0d0ece12edb07d65f39682217e3bad3): sharp_am: Build SAT (streaming aggregation) trees
- [#2137](https://github.com/Mellanox/sharp/pull/2137): shrarp_coll: enable/test V2 datatypes(16 bit)/ops(minloc/maxloc)
- [#2138](https://github.com/Mellanox/sharp/pull/2138): SHARPD: Support SAT QP configuration
- [#2135](https://github.com/Mellanox/sharp/pull/2135): Add flags to libsharp sharp_create_job
- [#2134](https://github.com/Mellanox/sharp/pull/2134): SHARPv2: Add peer tree
- [#2133](https://github.com/Mellanox/sharp/pull/2133): Update job data with endianness and reproducibility
- [#486273b](https://github.com/Mellanox/sharp/commit/486273b41646f8f8b5dfef57af39abbcb85269b0): SHARPv2: Add req_feature_mask to begin job message
- [#9115bb5](https://github.com/Mellanox/sharp/commit/9115bb59e33fcd117400642ab1587a769f9f085a): sharp_am: Fix Raven reproducible buffer calculation
- [#d71ef68](https://github.com/Mellanox/sharp/commit/d71ef6881f42c56db4b052ea77b1fe255806fa27): sharp_am: Fix setting data path version
- [#2124](https://github.com/Mellanox/sharp/pull/2124): SHARP V2 Headers
- [#7949054](https://github.com/Mellanox/sharp/commit/79490546cb2e9b2a91485f12c064c0e2f1f19b81): SHARPv2: Update AM tests with control_path_version field in JobData
- [#898a5ae](https://github.com/Mellanox/sharp/commit/898a5ae2663a1f134d731fe4fb833de2ec22493a): SHARPv2: Provide control path version in JobData
- [#c2a44cb](https://github.com/Mellanox/sharp/commit/c2a44cbeee86bd4d4e541de84a57e2a33dafe08e): sharp_am: Extend supported jobs number
- [#2119](https://github.com/Mellanox/sharp/pull/2119): Fix raven reproducible buffer calculation
- [#268c187](https://github.com/Mellanox/sharp/commit/268c187924a85c5820fc2846173abbef27e20d01): sharp_am: Support radix 80 (#2113)
- [#2122](https://github.com/Mellanox/sharp/pull/2122): SHARPv2: Fix passing capabilities to in sharp_get_tree_info
- [#ae78c68](https://github.com/Mellanox/sharp/commit/ae78c68ae3109ed73982cc6aa8a05c06f36e20da): SHARPv2: Add GroupJoin V2 MAD support to SHARPD
- [#cbd5e23](https://github.com/Mellanox/sharp/commit/cbd5e2307bc650a43563f9b4175c8f280fb482ed): SHARPv2: Add print MAD macros in SHARPD
- [#91dcefa](https://github.com/Mellanox/sharp/commit/91dcefa0c8ba6a5c36b55786f6a806d1834ddb6d): SHARPv2: Update generated MAD code used in SHARPD
- [#d8f634e](https://github.com/Mellanox/sharp/commit/d8f634e4fa367e027dcd9f3d9692537cf41377cd): SHARPv2: Use tree control path version from job info
- [#9b4f55d](https://github.com/Mellanox/sharp/commit/9b4f55d955ced63f49117236f455e1b3c34ac6dd): SHARPv2: Add management class version to MAD initialization functions
- [#b41f5e2](https://github.com/Mellanox/sharp/commit/b41f5e2f19fc80dfb39693ab2f245b9ad5ecb085): SHARPv2: Fix data type in group join v2 MAD function
- [#375c97e](https://github.com/Mellanox/sharp/commit/375c97e02e6b6ae63bae88e4e0da93f4ab6bbe74): SHARPv2: Remove unused MADs from SHARPD
- [#a589a8b](https://github.com/Mellanox/sharp/commit/a589a8b5f76d6a375cab4b36b1cce606b8d89201): sharp_am: Support 1K user_data
- [#6d4b997](https://github.com/Mellanox/sharp/commit/6d4b9973d58861b804cdc94d296c472bf54ffc82): sharp_am: Support V2 error traps
- [#34a9d6e](https://github.com/Mellanox/sharp/commit/34a9d6ec6909d7de97dca9f2dc4759929214d79a): sharp_am: Expand sharp_job_id_t to 32 bit
- [#281742e](https://github.com/Mellanox/sharp/commit/281742e96c6fd59dc2472f257f2a8b5498ced269): sharp_am: flip sharp_versio bit 0 on setting data_path_version
- [#9945b1e](https://github.com/Mellanox/sharp/commit/9945b1e6e1437eb864a8b3264cde5efb57049bd6): sharp_am: Update tree_info.feature_mask
- [#e5c46c7](https://github.com/Mellanox/sharp/commit/e5c46c70dc37d018b5ad2d8607a877e12b7408de): sharp_am: Set data path version
- [#3c3a17c](https://github.com/Mellanox/sharp/commit/3c3a17c7a2a2d821dbdc52d5ea4728fd720d8a0f): Refactoring: rename am_class_version to control_path_version
- [#670e89f](https://github.com/Mellanox/sharp/commit/670e89f43cd7e87fc7ed29dbbb05d88ae49f0ea8): sharp_am: Add control_path_version configuration parameter
- [#e5f13a5](https://github.com/Mellanox/sharp/commit/e5f13a5f77773a747bbd9e7042a944b63dee6fe8): sharp_am: Calculate is_eagle_calculation instead of using line_size
- [#f6c984d](https://github.com/Mellanox/sharp/commit/f6c984d9ed9f8e93607b5c6dc052487606211723): sharp_am: Read buffer calculation parameters from AnInfo
- [#0e82543](https://github.com/Mellanox/sharp/commit/0e82543b0b5cb411b5f10b35d041d2752d5a3fa8): sharp_am: Add Raven buffer calculation
- [#f8d8ff2](https://github.com/Mellanox/sharp/commit/f8d8ff234336c12732aa7c7052b6c1f5a6cc1fdf): sharp_am: Use AnInfo.tree_radix_used for buffer calculation
- [#2100](https://github.com/Mellanox/sharp/pull/2100): SHARPv2 API changes
- [#a5d841a](https://github.com/Mellanox/sharp/commit/a5d841a88606d778166d8f29a80c20dccb2770ab): sharp_am: Add AggNodeInfo dump to dump_fabric.txt
- [#f5c4d9a](https://github.com/Mellanox/sharp/commit/f5c4d9a8b5c403af81ff2036fdc5364e86f4b962): Support an class version 2
- [#2087](https://github.com/Mellanox/sharp/pull/2087): SHARPv2 - SHARPD MADs

### Protobuf based serialisation

- [#780183d](https://github.com/Mellanox/sharp/commit/780183d05387b8f990fbd4ec18c312d398131d21): SMX: link with protobuf statically (#2191)

### CUDA support

- [#2148](https://github.com/Mellanox/sharp/pull/2148): SHARP_COLL: CUDA support
- [#2144](https://github.com/Mellanox/sharp/pull/2144): sharp_coll: mpool utility for object pools

### Added

### Fixed
- [#4a47312](https://github.com/Mellanox/sharp/commit/4a47312412fbfbb0cb2806721081824edc9151dc): sharp_am: Fix timing of handling old jobs (#2198)
- [#03972fe](https://github.com/Mellanox/sharp/commit/03972fe025d05d373ff01ff315c9a5b43a8493fd): sharp_am: Fix Qp::Set invalid warning
- [#2195](https://github.com/Mellanox/sharp/pull/2195): sharp_coll: Fix num seg for zcopy recv
- [#bfacc78](https://github.com/Mellanox/sharp/commit/bfacc78f92f04e3660ff6a79884bf5216eff98f7): sharpd.c: Fix wrong condition when handling proxy join response (#2192)
- [#3cff055](https://github.com/Mellanox/sharp/commit/3cff055f927fc49d027ac168e7d84abf6f6db5f7): contrib/ufm_deployment.sh: Update UFM build script (#2182)
- [#acaccbb](https://github.com/Mellanox/sharp/commit/acaccbb94648cd66b4e8576744b90c1a82c8c762): ufm_deployment.sh: Enable external configuration for list of building servers
- [#2165](https://github.com/Mellanox/sharp/pull/2165): sharp_coll: fix ibv_create_qp return check
- [#2162](https://github.com/Mellanox/sharp/pull/2162): Makefile.am: Distribute CHANGELOG
- [#2151](https://github.com/Mellanox/sharp/pull/2151): smx: fix deadlock on smx_stop
- [#4ce6414](https://github.com/Mellanox/sharp/commit/4ce6414e0199e2cfe904bb3140aea74754d9ea32): sharp_am: Fix Raven reproducible buffer calculation tests
- [#2123](https://github.com/Mellanox/sharp/pull/2123): support non standard ipoib interface names(not ibX)
- [#b5a6836](https://github.com/Mellanox/sharp/commit/b5a683609c096d7b1fd603fe2a26c6c2d9d92ec3): AM: Change error message of using default configuration file to warning.
- [#5d8b58c](https://github.com/Mellanox/sharp/commit/5d8b58ce4b9bceceb67105c19387e1b12d7b0e48): sharp_am: Fix job data connections sorting (#2097)
- [#29c71ed](https://github.com/Mellanox/sharp/commit/29c71ed59a08890eea48bf97f09b4ec0a497d1cd): Link IBIS statically

### Changed

- [#f7f2483](https://github.com/Mellanox/sharp/commit/f7f24836bf7ea245248919d97e78efb8c6cadc05): sharp_helloc: remove hostname setting
- [#2167](https://github.com/Mellanox/sharp/pull/2167): sharp_coll/test: distribute sharp coll low level test with non-mpi build
- [#2177](https://github.com/Mellanox/sharp/pull/2177): Support gcc 8.2
- [#bde640f](https://github.com/Mellanox/sharp/commit/bde640fb92d0f375e26625b53a0c9c1a538e02d1): sharp_am: Log AM startup and shutdown to syslog

### New platforms

|Distro      |Platform |Kernel          |
|------------|---------|----------------|
|Fedora28    |x86-64   |3.10.0-693      |
|Ubuntu18.10 |x86-64   |3.10.0-693      |

### Changed

- [v1.7.2](https://github.com/Mellanox/sharp/commit/)
- [HEAD](https://github.com/Mellanox/sharp/commit/e5e5a243460ac80c3927e91612ead55de38dc99b)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.7.2...HEAD)


## v1.7.0

### Changed

- [#4ea3365](https://github.com/Mellanox/sharp/commit/4ea3365ff078acd27cf84b7a359a8fa4556cf864): sharp_am: Use new ibis api with version support
- [#00bd55d](https://github.com/Mellanox/sharp/commit/00bd55dc7c4e9bf6a935db2c805ddfb2715e578e): AM: allow running the AM on non-subnet manager node (#2071)
- [#c2df507](https://github.com/Mellanox/sharp/commit/c2df50708f7a47f440dace6bf58124ec5112f37d): sharpd: Remove rdma_init_counter from sharpd_job
- [#0ced73c](https://github.com/Mellanox/sharp/commit/0ced73c92f24657a0c95cea2e0d116fa64c9383a): Signal handler: Move signal handler to common (#2073)
- [#6887442](https://github.com/Mellanox/sharp/commit/6887442dfb6670b8a073c93747c87b87dece8754): allocated_quota_configuration documentation (#2018)

### Fixed

- [#2088](https://github.com/Mellanox/sharp/pull/2088): sharp_am: Check if SM is active when loading fabric information
- [#2081](https://github.com/Mellanox/sharp/pull/2081): shaprd_ops.c: Fix coverity reported issues
- [#a1b4d57](https://github.com/Mellanox/sharp/commit/a1b4d57e1427a614082ff7b87af714e4a9c0de31): libsharp_coll: Fix leakage (#2086)
- [#ea1e3d2](https://github.com/Mellanox/sharp/commit/ea1e3d2d06a4f8ac34e8a2644d1e87e417b41da8): sharpd: Fix invalid MGID when sharp_rdma_mcast_open fails
- [#cb5490e](https://github.com/Mellanox/sharp/commit/cb5490e08ebaddca88f752ce5abc20202f8a514c): sharpd: Fix data type for num_osts (#2070)
- [#641b3aa](https://github.com/Mellanox/sharp/commit/641b3aa24a12ae0af7ee1797813fbeee4832677c): libsharp_coll: Fix leakage (#2080)

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/8a86d1afcc1ff1769a99d19e644a63e4f394f633)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.6.0...8a86d1afcc1ff1769a99d19e644a63e4f394f633)


## v1.6.0

## Main features

### Job priority and accumulated quota limit for low-priority jobs

- [#2060](https://github.com/Mellanox/sharp/pull/2060): sharpd.h: Remove unused define
- [#df8cae3](https://github.com/Mellanox/sharp/commit/df8cae338d948a70f5262861d9affb6c71d66c10): jquota.c : Set maximum value for job priority
- [#6295461](https://github.com/Mellanox/sharp/commit/6295461bd09ab9b8b06d2e995dcb4397c7bcabca): sharpd.h,sharpd_ops.c: Check application provided priority
- [#286bd82](https://github.com/Mellanox/sharp/commit/286bd826e3c29c421e6e7bcb2b2e896fbe4fbbf7): JS API, SD: Set job priority
- [#c7439fe](https://github.com/Mellanox/sharp/commit/c7439fecc218ea87c81fa7d61745ef6a0c56d3c0): opts.c: Add SHARP_COLL_JOB_PRIORITY parameter
- [#453fd3b](https://github.com/Mellanox/sharp/commit/453fd3b36ca808496e259b13326ff33b0fb3f780): sharp_am: Limit job allocated quota by priority accumulated quota (#2019)

### Non-reproducible mode
- [#2065](https://github.com/Mellanox/sharp/pull/2065): sharp_am: Change default enable_reproducibility to TRUE
- [#c39c1ca](https://github.com/Mellanox/sharp/commit/c39c1cabce17fbf4c14adc3b472a5a4ce1088c5f): sharp_am: Support non reproducibility mode

### Percentage based resource allocation
- [#2064](https://github.com/Mellanox/sharp/pull/2064): sharp_coll: Add option to set osts per group
- [#55752b5](https://github.com/Mellanox/sharp/commit/55752b5998c9ddd145228641d288c7aceb7b90f3): sharp_am: Allocate quota by AN quota percent
- [#16e5b26](https://github.com/Mellanox/sharp/commit/16e5b26d7654981ebf4d7459784108b1b99b7e64): sharp_am: Change quota calculation (#2032)
- [#83fa354](https://github.com/Mellanox/sharp/commit/83fa3543957867a6bd42ac2a5f0b344a35c6300a): SD, API: Rename resource_percent to quota_percentage
- [#9ef0f94](https://github.com/Mellanox/sharp/commit/9ef0f9472521a3277d92ca18ec1718f8f2e85d11): sharpd.c: Transfer resource percentage to sharp_am
- [#03b8304](https://github.com/Mellanox/sharp/commit/03b83046a75f0e0c90c8630736963152406d7759): SD, API: Add percent of SHARP resources to JS API
- [#43e6e2b](https://github.com/Mellanox/sharp/commit/43e6e2bbfaa6ed7c197d89265e694f8eb717cb49): sharp_am: Support allocating resources according to percentage
- [#6813fa6](https://github.com/Mellanox/sharp/commit/6813fa6fcaeb4fa083369c1361a53c8839fcc1a0): SMX: Add tests for begin job with quota percentage
- [#4e77d19](https://github.com/Mellanox/sharp/commit/4e77d19253441296706bce74265aa5f705cda66e): SMX: Add quota request in percentage to sharp_begin_job message

### Fixes

- [#c0c7841](https://github.com/Mellanox/sharp/commit/c0c784116bea1b6e9278aabb68132c2da5850a6e): sharp_am: Remove error msg for vports (#2059)
- [#44218c2](https://github.com/Mellanox/sharp/commit/44218c28089246cec069f84fe9d0392a6391d2a9): sharpd_ops.c: Fix log messages
- [#2036](https://github.com/Mellanox/sharp/pull/2036): sharp_benchmarks2.sh: use ucx for mpi default transport
- [#3019f70](https://github.com/Mellanox/sharp/commit/3019f70b23b35b9e8257bebf16c673dcc28e7492): sharp_am: use timeout and retires from configuration file (#2031)
- [#37b922f](https://github.com/Mellanox/sharp/commit/37b922f634d34a62abb8bdc4afb89660bab8f42f): sharpd: Fix non-matchin parsing functions
- [#a066d97](https://github.com/Mellanox/sharp/commit/a066d977bb3c511563138e848cfc069dd0fce617): sharp_opt_parser.h: Add builtin parsing functions
- [#85a0b99](https://github.com/Mellanox/sharp/commit/85a0b99699caefff7fe479b6015e27fc381b2e9c): libsharp/Makefile.am: Remove dependency on libsmx
- [#823db7e](https://github.com/Mellanox/sharp/commit/823db7e3e0674015835339767f8cd9c5b798c0b0): sharp_am: Fix option_info overflow by (#2030)
- [#c70e564](https://github.com/Mellanox/sharp/commit/c70e564d629d91b118fba5067da59633b697a288): smx: Add name to enum data types
- [#2024](https://github.com/Mellanox/sharp/pull/2024): Fix clang compilation errors
- [#2021](https://github.com/Mellanox/sharp/pull/2021): Fix clang compilation errors
- [#ad02520](https://github.com/Mellanox/sharp/commit/ad025208fc369bdef0ae9218d01438255c90c6a4): doc/JobTreesConfiguration.md: Add document
- [#48c4ebb](https://github.com/Mellanox/sharp/commit/48c4ebb912131da6fcceeaa5b5c16c450e35c400): sr.c: Write warning when finding service record of different port GID (#2007)
- [#bf98580](https://github.com/Mellanox/sharp/commit/bf985806604ca9a1c39422d22a9eaceaefb9e129): sharp_manager.sh: Avoid using SSH to local host (#2017)
- [#0d30679](https://github.com/Mellanox/sharp/commit/0d3067978be3639222cb834addcf45bd6a43b1af): sharp_am: Support DFP with leafs with no AN attached (#2012)
- [#e1b29ef](https://github.com/Mellanox/sharp/commit/e1b29ef6c0c44c19e5346d5ff1036943222b7c8c): sharp_am: start job handling after parsing all fabric files (#1976)
- [#83dc441](https://github.com/Mellanox/sharp/commit/83dc441a896d7a928150fe355fe490d4ea734438): sharp_benchmark2.sh: skip cuda version of osu benchmarks
- [#651f772](https://github.com/Mellanox/sharp/commit/651f772346c4bc28fef30f45e4874e4e73de03b3): sharp_manager.sh: Change umask
- [#267296e](https://github.com/Mellanox/sharp/commit/267296e51c2aeea4480692ea37bc27c2ff546bba): sharpd: Add port number in sharp_get_tree_connect_info request (#1993)

### Changed

- [#2057](https://github.com/Mellanox/sharp/pull/2057): README: Add RHEL 7.5 to list of supported platforms
- [#2063](https://github.com/Mellanox/sharp/pull/2063): V1.6 improve smx logging

### Changes in parameters

|Parameter                      |Component|  Status     |  Description                          |
|-------------------------------|---------|-------------|---------------------------------------|
| --max_tree_radix              | AM      |  Changed    |The value should be a multiple of four.|
| --default_quota               | AM      |  Deprecated | Replaced by per_prio_default_quota    |
| --max_quota                   | AM      |  Deprecated | Replaced by per_prio_max_quota        |
| --per_prio_max_quota          | AM      |  New        | Maximum percentage of quota (OSTs, Buffers and Groups) per aggregation node per tree, that can be requested by a single job by its priority.|
| --per_prio_default_quota      | AM      |  New        | Default percentage of quota (OSTs, Buffers and Groups) per aggregation node per tree, to be requested for a single job by its priority.|
| --low_prio_max_accumulated_quota| AM      |  New        | Maximum accumulated quota (OSTs, Buffers and Groups) percentage that can be allocated for all low priority jobs (priority = 0) on a single AN.|
| --max_trees_per_job            | AM      |  New        | Maximum number of trees per job It is guarantee that no job will receive more than max trees |
| --default_trees_per_job        | AM      |  New        | Default number of trees per job The number of trees allocated for a job if it was not requested explicitly |
| --max_compute_ports_per_agg_node| AM      |  New        | Maximum number of compute ports connected to the same aggregation node. |
| --job_priority                  | JS      |  New        | Job priority                                                            |
| --coll_job_quota_percentage     | JS      |  New        | Sets upper limit of SHARP resources available for the job               |


### Changes in public API

#### JS API

|Parameter                    | Default value | Description                                 |
|-----------------------------|---------------|---------------------------------------------|
| --job_priority              | 0             | Sets priority of the job [0..9]             |
| --coll_job_quota_percentage | 0             | Sets upper limit of SHARP resources available for the job. Default value 0 means, use sharp_am default according to priority.|          


### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/e980f39d801b94805348034889e4b9d6a4b041be)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.6.0-rc...e980f39d801b94805348034889e4b9d6a4b041be)


## v1.5.3

### Fixed
- [#1997](https://github.com/Mellanox/sharp/pull/1997): Fix issues in sharp_manager.sh and sharp.init.in
- [#1999](https://github.com/Mellanox/sharp/pull/1999): Fix handling job resources allocation when number of rails is larger than number of trees
- [#2000](https://github.com/Mellanox/sharp/pull/2000): sharp_coll: Fix port_idx when (num_tress * max_group_channles) greater than num_ports

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/ce747964e57e54175e2a61661554e8516b1748e4)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.5.2...ce747964e57e54175e2a61661554e8516b1748e4)

## v1.5.2

### Fixed

- [#e7ab6fd](https://github.com/Mellanox/sharp/commit/e7ab6fdbf71768da543ea81e9bd755c1928086c6): sharpd: Fix groups num idx exceeds groups num size

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/e7ab6fdbf71768da543ea81e9bd755c1928086c6)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.5.1...e7ab6fdbf71768da543ea81e9bd755c1928086c6)

## v1.5.1

### Fixed

- [#1991](https://github.com/Mellanox/sharp/pull/1991): smx/Makefile.am: Change libsmx naming convention
- [#1989](https://github.com/Mellanox/sharp/pull/1989): sharp_coll: Fix warning on Fedora 23 ( gcc 5.1.1-4)

## v1.5.0

## Main features

### DF+

- [#1937](https://github.com/Mellanox/sharp/pull/1937): sharp_am: Support Dragonfly topology

### Multi-rail

- [#b8d6474](https://github.com/Mellanox/sharp/commit/b8d64740d7c46ab770163abf5f500612a0402f8d): sharpd:Fix leave mcast error - leaving mcast address should be in network order (#1980)
- [#a8f60f9](https://github.com/Mellanox/sharp/commit/a8f60f9c716538d2b0ab6bd8e53366511038901e): Multirail: Support additional job tree configurations
- [#dc96b73](https://github.com/Mellanox/sharp/commit/dc96b7347ce9ef5e1882b142db97e139ac347fc0): sharpd.c: Fix infinite loop when multicast disabled
- [#fe7a973](https://github.com/Mellanox/sharp/commit/fe7a97349067b46e9c8c88c91bd53ca268294457): sharpd: Fix error in multi-subnet when am can't be reached
- [#a8f60f9](https://github.com/Mellanox/sharp/commit/a8f60f9c716538d2b0ab6bd8e53366511038901e): Multirail: Support additional job tree configurations
- [#570879a](https://github.com/Mellanox/sharp/commit/570879a400de37cdb0ac0e54ab2623eb6ad173da): sharp_coll: set PROCESSES_DATA as default job member list type
- [#8097119](https://github.com/Mellanox/sharp/commit/80971197e0b886bd62e65cf9a409b6526f7ecaca): sharpd: Move device context from sharp_tree to sharpd_tree_conn
- [#1903](https://github.com/Mellanox/sharp/pull/1903): Client multirail full includes multicast changes
- [#09127f3](https://github.com/Mellanox/sharp/commit/09127f370e465cdd211b2941c14b9cf9d39dbf59): sharpd: Add aggregation nodes list per sharpd tree
- [#d0f1382](https://github.com/Mellanox/sharp/commit/d0f138293c440bf30d1a61d1c26fdb07bd80c98c): sharpd: Add support for multirail - Create begin_job matrix
- [#1884](https://github.com/Mellanox/sharp/pull/1884): sharp_coll: connect multiple trees on different rails
- [#20b4923](https://github.com/Mellanox/sharp/commit/20b49235b8fecebd36b8fa30fb2b6ea33bfa4258): sharpd: Fix alloc group crash
- [#66e63bd](https://github.com/Mellanox/sharp/commit/66e63bd760d7fcff64b7fc4ce6b2c6e0657a8802): sharpd.c: Remove one-group limitation at sending group allocation request to SD#0
- [#db20e62](https://github.com/Mellanox/sharp/commit/db20e62bddd33424efbdb4f805886ca9e4e63eb1): smx_types.h: Add SHARP_INVALID_TREE_ID
- [#dab2233](https://github.com/Mellanox/sharp/commit/dab2233ccf258cc4c2ce8bd9c59eb30afb141949): sharpd: Add find_sharpd_tree_by_tree_idx function
- [#35cf3f4](https://github.com/Mellanox/sharp/commit/35cf3f4108f9a602129b892160717ff66e8cdf80): sharpd: Rename find_sharpd_tree function
- [#bb99551](https://github.com/Mellanox/sharp/commit/bb99551bc11cce0773f2c35403c54ae46a4502f6): AM: add multirail support (#1788)
- [#1860](https://github.com/Mellanox/sharp/pull/1860): sharpd.c: Remove one-group, single-tree limitation
- [#d6ff1fe](https://github.com/Mellanox/sharp/commit/d6ff1fee5de5429ed79eb1ad87aeb140eee32f29): libsharp: Add array of tree indexes to sharp_alloc_groups_info
- [#32313d0](https://github.com/Mellanox/sharp/commit/32313d08d6ba3a89152bd57292e53f9224a6c556): SMX: Clean smx_group_data
- [#6959c69](https://github.com/Mellanox/sharp/commit/6959c6955e56e1ab337ad9c41eb5b3622ff9ab36): SMX: Remove GROUP_DATA as SMX message
- [#1815](https://github.com/Mellanox/sharp/pull/1815): SMX: Replace SHARP_MSG_TYPE_ALLOC_GROUP by SHARP_MSG_TYPE_ALLOC_GROUPS
- [#1817](https://github.com/Mellanox/sharp/pull/1817): SMX: Replace SHARP_MSG_TYPE_RELEASE_GROUP by SHARP_MSG_TYPE_RELEASE_G…

### Support for virtual ports

- [#1982](https://github.com/Mellanox/sharp/pull/1982): Fix invalid port fabric dump
- [#01c2689](https://github.com/Mellanox/sharp/commit/01c268940272734187691616e394af915de0fd72): tests: Add virtual ports tests (#1978)
- [#1940](https://github.com/Mellanox/sharp/pull/1940): Add virtual port support
- [#1947](https://github.com/Mellanox/sharp/pull/1947): AM: Validate LIDs
- [#1946](https://github.com/Mellanox/sharp/pull/1946): AM: Add parameter for virtual ports file path

### Support for SM switch reboot

- [#1941](https://github.com/Mellanox/sharp/pull/1941): Add timestamp and AN rediscovery support to AM

## Unreleased feature

### Topology aware resource allocation

- [#d55d418](https://github.com/Mellanox/sharp/commit/d55d4185a7841f47d3436292f6a3c33ed87146af): Refactor sharpd group allocation code (#1935)
- [#1931](https://github.com/Mellanox/sharp/pull/1931): Add resources to sharpd_tree_an Add feature_mask to sharpd_tree
- [#bee5e85](https://github.com/Mellanox/sharp/commit/bee5e85b3cb55e90308235e59f4fe0afefc6bca9): sharpd: update group_join_mad
- [#1924](https://github.com/Mellanox/sharp/pull/1924): sharp_coll: group trim updates
- [#1921](https://github.com/Mellanox/sharp/pull/1921): Add group trimming mode
- [#58cdf8f](https://github.com/Mellanox/sharp/commit/58cdf8f4df4f7ce5609c9977ddb526975980d606): sharp smx: Add processes in group to sharp_alloc_group (#1816)
- [#1845](https://github.com/Mellanox/sharp/pull/1845): sharp_coll: Add process node local rank to sharp context
- [#1844](https://github.com/Mellanox/sharp/pull/1844): sharp_coll API: Add process node local rank to sharp_coll_init

## API changes

|Function                     |Change              |Related features                                   |
|-----------------------------|--------------------|---------------------------------------------------|
|sharp_get_local_data         |Added               |Multi-rail,Topology aware resource allocation      |
|sharp_release_local_data     |Added               |Multi-rail.Topology aware resource allocation      |
|sharp_release_local_data     |Added               |Multi-rail,Topology aware resource allocation      |
|sharp_get_tree_connect_info  |Behaviour changed   |Multi-rail                                         |
|sharp_alloc_groups_info      |Added               |Multi-rail                                         |
|sharp_release_groups_info    |Added               |Multi-rail                                         |
|sharp_alloc_group_info       |New parameters      |Multi-rail,Topology aware resource allocation      |
|sharp_create_job             |New parameters      |Multi-rail,Topology aware resource allocation      |

## Fixed

- [#1987](https://github.com/Mellanox/sharp/pull/1987): sharp_am: Add port key to error messages
- [#1981](https://github.com/Mellanox/sharp/pull/1981): sharp_daemons_setup.sh: Fix systemd files installation dir
- [#1974](https://github.com/Mellanox/sharp/pull/1974): sharp_coll: Fix cleanup when create job is failed
- [#c8ac33a](https://github.com/Mellanox/sharp/commit/c8ac33a1464c42e9e197718082d0119bbb955736): sharp_am: Handle MAD send temporary errors on fabric_graph (#1971)
- [#946e2ef](https://github.com/Mellanox/sharp/commit/946e2efe28093df6154719be458a59441a49d356): sharp_am: Fix generation of fabric dump file when option disabled
- [#1977](https://github.com/Mellanox/sharp/pull/1977): sharp_am: Fix job id leakage
- [#bb2498e](https://github.com/Mellanox/sharp/commit/bb2498eae3704d0ff9c55afbfbc234922dd681b6): sharp_am: Fix Fabric update: New AN on existing SW (#1952)
- [#92ccb58](https://github.com/Mellanox/sharp/commit/92ccb584a1218aff20aaa874029528cdef188bf8): sharp_am: Fix crossng cables on fabric update (#1938)
- [#1943](https://github.com/Mellanox/sharp/pull/1943): Update SHARPD MADs timeout/retries polocy
- [#1923](https://github.com/Mellanox/sharp/pull/1923): sharpd: Fix static QoS
- [#9e8849e](https://github.com/Mellanox/sharp/commit/9e8849ef0758f13b8fd29cf6eb7e5e5d3055dd69): systemd: Fix sharp_am configuration as systemd service
- [#ad29c52](https://github.com/Mellanox/sharp/commit/ad29c52b79ad98521b5eae67b9d2813add781616): sharpd: fix local endpoint address published by master SD
- [#c278d4b](https://github.com/Mellanox/sharp/commit/c278d4b1fa95ec073eefa1b3b8858a6c670d757b): smx: fix printing error message on smx_addr_ep2str failure
- [#1901](https://github.com/Mellanox/sharp/pull/1901): sharp_am: Fix memory leak when writing MAD messages to log file
- [#1894](https://github.com/Mellanox/sharp/pull/1894): sharp_coll: Fix non blocking fragments ordering
- [#2675ee3](https://github.com/Mellanox/sharp/commit/2675ee3931ca0923203fcba60315d93b1cf07fa6): SBA: Fix sharp_daemons_setup.sh usage message
- [#9c7b6f0](https://github.com/Mellanox/sharp/commit/9c7b6f08eb7d40b09a06a370dd5df7b7453bec13): SBA: Do not start sharpd service automatically on boot
- [#5678ef8](https://github.com/Mellanox/sharp/commit/5678ef8fd94eb1d245772c5e9bdd0b56f205defe): sharpd: Fix writing SMX log messages to log file
- [#1863](https://github.com/Mellanox/sharp/pull/1863): sharp_coll: Fix non-blocking colls
- [#1854](https://github.com/Mellanox/sharp/pull/1854): Fix systemd binaries location and add optional configuration file parameter.
- [#4eeb90e](https://github.com/Mellanox/sharp/commit/4eeb90e9dde19c3da6208c4e9c6c786bc1e40172): SBA: Remove error message when stopping a service that is not loaded
- [#1847](https://github.com/Mellanox/sharp/pull/1847): sharp_coll: bcast job info status in a structure
- [#1846](https://github.com/Mellanox/sharp/pull/1846): sharp_coll: move oob barrier before calling sharp_get_tree_info

## Optimizations

- [#1949](https://github.com/Mellanox/sharp/pull/1949): sharp_coll: minor code optimizations
- [#1920](https://github.com/Mellanox/sharp/pull/1920): sharp_am: consider number of ANs in tree when comparing trees
- [#3d23461](https://github.com/Mellanox/sharp/commit/3d23461faaef983c9d360ebf6d70e2c458abc054): sharp_coll: push job data from only process on a node
- [#1848](https://github.com/Mellanox/sharp/pull/1848): sharp_coll: optimize max_ppn calculation using node local ranks

## Parameters

- [#570879a](https://github.com/Mellanox/sharp/commit/570879a400de37cdb0ac0e54ab2623eb6ad173da): sharp_coll: set PROCESSES_DATA as default job member list type
- [#1883](https://github.com/Mellanox/sharp/pull/1883): sharp_coll: increase default buf pool size.

### Changes in parameters

|Parameter                      |Component|Description                                                                                                          |
|-------------------------------|---------|---------------------------------------------------------------------------------------------------------------------|
|-O, --config_file              | AM / SD | If specified with '-' prefix, ignore configuration file read errors and used default configuration file. Exit if '-' is not specified and fails to read configuration file.|                                                                                                  |

## Scripts

- [#5332308](https://github.com/Mellanox/sharp/commit/533230890ef9b80fcd8cdbe4393790e9e5535303): sharp_benchmark2.sh: Set IB device for UCX
- [#05b54f0](https://github.com/Mellanox/sharp/commit/05b54f08f2b5c6e088d27023b4317de2ee3fa8ed): sharp_benchmark2.sh: Enable SHARP for 4K messages
- [#2d05b71](https://github.com/Mellanox/sharp/commit/2d05b7181c83bfb5d5713bc8a42d1c622f70f7d1): sharp_benchmark2.sh: Propagate LD_LIBRARY_PATH to nodes manually (#1969)
- [#7980fd7](https://github.com/Mellanox/sharp/commit/7980fd7743e430f830140e52e491de1e8a13ec7e): Adding disable-systemd in the ufm build script. (#1951)
- [#65253c8](https://github.com/Mellanox/sharp/commit/65253c8cce01bdd7fff1e4e3d93a2310b685da59): sharp_benchmark.sh: Use --report-bindings instead of --display-map
- [#1907](https://github.com/Mellanox/sharp/pull/1907): sharp_manager.sh: Remove "sudo" requirement for sharp_manager.sh

## Internal changes

- [#1984](https://github.com/Mellanox/sharp/pull/1984): AM tests: Change am tests dump files without apostrophes
- [#b3ed63b](https://github.com/Mellanox/sharp/commit/b3ed63bba7a4d4970b7a1c7db20e8d2bea8883e5): .gitmodules: Switch to sharp-v1.5.0 branch in ibis
- [#6facbf1](https://github.com/Mellanox/sharp/commit/6facbf1fac22ad9c78938e0209caa15dee6e3e81): Update copyright
- [#57bb3ec](https://github.com/Mellanox/sharp/commit/57bb3ec3b8cbaff7268cdf68d99d1f78e0db9b74): sharp_am: Add thread pool and parallel tree finder (#1738)
- [#1926](https://github.com/Mellanox/sharp/pull/1926): sharp_coll: MPI_THREAD_MULTIPLE support with non-blocking API
- [#1918](https://github.com/Mellanox/sharp/pull/1918): sharp_coll: move pending coll list from context to comm struct
- [#1916](https://github.com/Mellanox/sharp/pull/1916): sharp_coll: thread multiple support

### Links
- [v1.5.0](https://github.com/Mellanox/sharp/commit/1ce5e19dca1f1c5157396933ca41769d5f185d52)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.4.4...1ce5e19dca1f1c5157396933ca41769d5f185d52)

## v1.4.0

### Main features

#### Fabric extension (non-root ANs)

- [#1723](https://github.com/Mellanox/sharp/pull/1723): Tree update

#### Optimized root placement

- [#1702](https://github.com/Mellanox/sharp/pull/1702): sharp_am: use_root_groups_in_build_trees

#### Sending ERROR message to running jobs at AM restart

- [#11b8799](https://github.com/Mellanox/sharp/commit/11b8799c7b2d8ff4c79dd2ff046ba92858f6f6c7): AM: send job error on AM restart.
- [#1a0c77f](https://github.com/Mellanox/sharp/commit/1a0c77f615a8f480fbe272156412ba87ac92fc6e): AM: Add hidden flag for disabling JOB ERROR on am restart

#### Systemd support and on-demand sharpd actication

- [#913bf98](https://github.com/Mellanox/sharp/commit/913bf9848186fb65635b586917dc058e07c35dc7): SHARP: Add systemd support for "sharpd" and "sharp_am" services
- [#9238d60](https://github.com/Mellanox/sharp/commit/9238d601a16c3e9c15ba03a7337756826b4dbeb3): src/common/log.c: Add syslog support (#1669)
- [#7064a33](https://github.com/Mellanox/sharp/commit/7064a33799d18e7c115674d4c228f842562f079a): sharpd_server.c: Add support for socket based activation
- [#ae7936f](https://github.com/Mellanox/sharp/commit/ae7936f914cb3af69dab968f2473c825a4731fce): configure.ac: Check SHARP is compiled in systemd based distro

### Future features

#### Multi-rail

- [#1712](https://github.com/Mellanox/sharp/pull/1712): sharp_coll: multiple device contexts
- [#1708](https://github.com/Mellanox/sharp/pull/1708): sharp_am: Allocate distinct trees

### Fixed

- [#1739](https://github.com/Mellanox/sharp/pull/1739): sharp_am: use kill instead of killpg to terminate the AM
- [#2f190e7](https://github.com/Mellanox/sharp/commit/2f190e7a6b1b19df8655b465847da44e85fd2752): Add support for retry/timeout parameters for opening the smdb/lst files
- [#1719](https://github.com/Mellanox/sharp/pull/1719): Avoid log file truncation when restarting AM
- [#1729](https://github.com/Mellanox/sharp/pull/1729): hostlist.c: Fix gcc 7.1.1 compilation warning
- [#1728](https://github.com/Mellanox/sharp/pull/1728): Logging optimization
- [#1709](https://github.com/Mellanox/sharp/pull/1709): sharpd: Fix sharpd listening socket name
- [#1703](https://github.com/Mellanox/sharp/pull/1703): AM: Add graceful exit when failing to open lst/smdb files
- [#17611c5](https://github.com/Mellanox/sharp/commit/17611c5da3a0b640687b2b8acd7ec3846cc37d4c): AM: Better handling for IB EventListener error.

### Changed

- [#5e4635a](https://github.com/Mellanox/sharp/commit/5e4635afa157656221766e4b84e261bae4b244fb): api/sharp_coll.h, api/sharp.h - Rename
- [#1696](https://github.com/Mellanox/sharp/pull/1696): sharp_am: Add multi trees support

### Workaround

- [#1736](https://github.com/Mellanox/sharp/pull/1736): configure.ac: Remove socket-based integration

### New platforms

|Distro      |Platform |Kernel          |
|------------|---------|----------------|
|RHEL 7.4    |x86-64   |3.10.0-693      |

#### New parameters

|Parameter                      |Component|Description                                                                                                                 |
|-------------------------------|---------|----------------------------------------------------------------------------------------------------------------------------|
| --accumulate_log              | AM/SD   | Accumulate log file over multiple sessions. If set to FALSE and log rotation is disabled, log file is truncated on startup.|
| --syslog_verbosity            | AM/SD   | Syslog verbosity level: 1 - Errors, 2 - Warnings. Default value is "1"                                                     |
| --persistent_dir              | AM      | Path to persistent data directory                                                                                          |

### Links

- [HEAD](https://github.com/Mellanox/sharp/commit/e0e10e0d69c2420126549b61fbbd63512a66d302)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.3.0...e0e10e0d69c2420126549b61fbbd63512a66d302)

## v1.3.1

### Fixed

- [#1678](https://github.com/Mellanox/sharp/pull/1678): sharpd.c: Fix fd connection leak
- [#1677](https://github.com/Mellanox/sharp/pull/1677): sharp_coll: Fix invalid sharp group cached pointer

### Links

- [v1.3.0](https://github.com/Mellanox/sharp/commit/a3525802039513026e654dadd76454cdc51c5385)
- [HEAD](https://github.com/Mellanox/sharp/commit/2f190e7a6b1b19df8655b465847da44e85fd2752)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.3.0...2f190e7a6b1b19df8655b465847da44e85fd2752)


## v1.3

### Main features

#### Support for extended LST format (needs SM v4.9 or later)

- [#7a9ca10](https://github.com/Mellanox/sharp/commit/7a9ca10ff5c7666b0d5aaea1d693fd06f9046469): sharp_am: Add load fabric from SMDB (csv format) file
- [#1620](https://github.com/Mellanox/sharp/pull/1620): sharp_am: move csv functions to FabricFileParser

#### Support multi-rail (removing ib_dev parameter)

- [#1645](https://github.com/Mellanox/sharp/pull/1645): Fix sd crash and synchronisation issues
- [#12ae613](https://github.com/Mellanox/sharp/commit/12ae613fa798054b771218569d70ed3923a05d11): Move port handling to scope of job (#1617)
- [#e4db25f](https://github.com/Mellanox/sharp/commit/e4db25fa4a17ade28d5cc978edc8c3a3e36eb998): sharpd.[c,h]: Move mcast leave to the main thread
- [#d2980fc](https://github.com/Mellanox/sharp/commit/d2980fccddf6f196ae449133489f7a83ae59008e): sharpd.c: Forward ud mcast request to AM as is

#### Handling adding/replacing compute node

- [#2edc2e4](https://github.com/Mellanox/sharp/commit/2edc2e4ab18d8f386c77b0a1382c870797cfd1b5): sharp_am: Support adding or moving compute ports

#### Management scripts

- [#1604](https://github.com/Mellanox/sharp/pull/1604): Add "-x" option for var. removing
- [#1603](https://github.com/Mellanox/sharp/pull/1603): Change SHArP to SHARP
- [#1602](https://github.com/Mellanox/sharp/pull/1602): Fix #1019188: exit code zero when install/unisntall failes on hosts
- [#1601](https://github.com/Mellanox/sharp/pull/1601): Add correct status exit code
- [#1600](https://github.com/Mellanox/sharp/pull/1600): Improve manager
- [#1595](https://github.com/Mellanox/sharp/pull/1595): Set env. vars. top priority
- [#1594](https://github.com/Mellanox/sharp/pull/1594): Fix #1017805: exit code zero on sharp manager start
- [#1593](https://github.com/Mellanox/sharp/pull/1593): Read data from cfg file before reading it from command line

#### Support for configuration parameters update in runtime

- [#1549](https://github.com/Mellanox/sharp/pull/1549): option update in runtime: Enable in SD, Fix typos
- [#1546](https://github.com/Mellanox/sharp/pull/1546): AM: Allow update of some parameters during runtime

#### Backporting for IPv4 support from v1.2.1

- [#1575](https://github.com/Mellanox/sharp/pull/1575): Ip opts v1.3

#### Span all ANs

- [#1577](https://github.com/Mellanox/sharp/pull/1577): sharp_am: Generate trees that span all ANs by default

### Removed features

- [#1630](https://github.com/Mellanox/sharp/pull/1630): sharp.c: Disable SM data distribution feature in API

### Internal changes

- [#5be0638](https://github.com/Mellanox/sharp/commit/5be06380ca1c9cb90c0ce911526e9a5e2d92cc75): sharp_am: Add is_valid to agg node dump
- [#f2a6091](https://github.com/Mellanox/sharp/commit/f2a609126564cc3ebc148408e1a30421b02adb64): sharp_am: Add error message when parsing rank/coordiante with invalid value fomr smdb file
- [#02b2beb](https://github.com/Mellanox/sharp/commit/02b2beb673c10ba4ce7063b1e909b61bb06dc35f): Makefile.am: Create sharp_benchmark.sh as a soft link to sharp_benchmark2.sh
- [#fb1f940](https://github.com/Mellanox/sharp/commit/fb1f94011c3ff22617224c05824e987437e312d0): benchmark commandline simplification (#1636)
- [#44029ec](https://github.com/Mellanox/sharp/commit/44029ec8e088db16a677df6947f51d2bf34ef95b): sharp_am: Add ExitAggMgr
- [#1630](https://github.com/Mellanox/sharp/pull/1630): sharp.c: Disable SM data distribution feature in API
- [#1609](https://github.com/Mellanox/sharp/pull/1609): sharp_coll: allow multiple segments to register with sharp coll.
- [#95c68db](https://github.com/Mellanox/sharp/commit/95c68db336a297886f010c235b761590dbe09404): sharp_am: Add multicast debug log
- [#024392f](https://github.com/Mellanox/sharp/commit/024392feb9cff177c7db2c9b8e9a64cc0d4883d2): sharp_gen_changelog.sh: Update the main CHANGELOG
- [#1592](https://github.com/Mellanox/sharp/pull/1592): SD: Make the ib_dev parameter deprecated
- [#1583](https://github.com/Mellanox/sharp/pull/1583): sharp_coll: Fix sharp stats header file include
- [#1567](https://github.com/Mellanox/sharp/pull/1567): SD: Improve handling of ports and devices
- [#1570](https://github.com/Mellanox/sharp/pull/1570): Print cmd line
- [#1568](https://github.com/Mellanox/sharp/pull/1568): Makefile.am: Remove sm_data_transfer_test from distribution
- [#1565](https://github.com/Mellanox/sharp/pull/1565): option_parser: Add new flag for deprecated params
- [#1429](https://github.com/Mellanox/sharp/pull/1429): Add stat collection to SHArPD
- [#1558](https://github.com/Mellanox/sharp/pull/1558): Fix build on ARM64
- [#1550](https://github.com/Mellanox/sharp/pull/1550): Improve IBIS logging
- [#bb509b5](https://github.com/Mellanox/sharp/commit/bb509b5d5dbf08d7a5a35f78c7cf46ded098a4e6): sharp_am: Delete job after begin job error
- [#1545](https://github.com/Mellanox/sharp/pull/1545): smx_sock.c: Remove "enter/leave" logging in SMX
- [#1542](https://github.com/Mellanox/sharp/pull/1542): AM: Change ib_mad parameters to hidden
- [#1540](https://github.com/Mellanox/sharp/pull/1540): sharp_coll: Fix api notes
- [#1534](https://github.com/Mellanox/sharp/pull/1534): remove workaround to avoid hcoll overhead
- [#1518](https://github.com/Mellanox/sharp/pull/1518): sharp_coll_test: make independent utility
- [#1510](https://github.com/Mellanox/sharp/pull/1510): SD: Remove redundant field from sharp job struct
- [#1521](https://github.com/Mellanox/sharp/pull/1521): SMX: Expand error-logging in pack/unpack

### Verification

- [#a4fa37a](https://github.com/Mellanox/sharp/commit/a4fa37a62c07e348026bb628d9200ece9b3ce00a): valgrind - Remove an unnecessary suppression
- [#4045e2d](https://github.com/Mellanox/sharp/commit/4045e2daeca11d8e1571ccdb5d99afe0a450976f): Add mpi test ability to Jenkins build

### Fixed

- [#e366324](https://github.com/Mellanox/sharp/commit/e366324023685ab624557d72108710ba84737f73): sharp_am: Fix ExitAggMgr
- [#cb2aed5](https://github.com/Mellanox/sharp/commit/cb2aed5185031adb5bc3f37647196d2dbce01411): sharp_coll: Fix a hang in non-blocking allreduce
- [#1660](https://github.com/Mellanox/sharp/pull/1660): sharp.spec.in: Suppress errors in rpm uninstall
- [#9b1c98a](https://github.com/Mellanox/sharp/commit/9b1c98af46bb6859a7b549c208886afea639a6b6): AM: Handle qp_child_idxs when logging (#1505)
- [#7786843](https://github.com/Mellanox/sharp/commit/77868434c218e777a5aa3ec645ce4ef7a2b1c449): sharp_am: Fix QP Recovery.
- [#0d365e1](https://github.com/Mellanox/sharp/commit/0d365e1d507cd216586eb2d9e77514e50be7151c): sharp_am: Handle AddLink Failure.
- [#1585](https://github.com/Mellanox/sharp/pull/1585): Fix: #1010204 & #1010188
- [#2da4610](https://github.com/Mellanox/sharp/commit/2da46105d10d7158b0e67b6947ca06b8d3da8a77): sharp_am: Fix is_pure FAT tree calculation.
- [#1523](https://github.com/Mellanox/sharp/pull/1523): SD: Do not send response if socket is invalid
- [#1504](https://github.com/Mellanox/sharp/pull/1504): Option parser - Bug fix and log verbosity tweak
- [#1515](https://github.com/Mellanox/sharp/pull/1515): sharp_hello: use random generated job ID
- [#1524](https://github.com/Mellanox/sharp/pull/1524): Correct "Usage" of sharp_daemons_setup.sh

### Changes in parameters

#### New parameters

|Parameter                      |Component|Description                                                                                                          |
|-------------------------------|---------|---------------------------------------------------------------------------------------------------------------------|
|span_all_agg_nodes             | AM      | Generate trees that span all possible aggregation nodes. Relevant only if "topology_type" is tree. Default is "true"|

#### List of parameters that could be updated in runtime

|Parameter                      |Component|Description                                                                                                          |
|-------------------------------|---------|---------------------------------------------------------------------------------------------------------------------|
|log_verbosity                  | AM / SD | Log verbosity level                                                                                                 |
|ftree_ca_order_file            | AM      | Path of ftree CA order file generated by OpenSM. Its contents can be used when implementing all-to-all communication|
|lst_file_timeout               | AM      | Length of timeout [in seconds] between attempts to load the LST file                                                |
|lst_file_retries               | AM      | Max number of retry attempts when loading the LST file and encountering "No such file" errors                       |
|generate_dump_files            | AM      | Dump internal state to files for debug and diagnostics                                                              |
|max_quota                      | AM      | Maximum quota that can be requested by a single job                                                                 |
|default_quota                  | AM      | Default quota to be requested for a single job                                                                      |

#### Added/Removed/Deprecated parameters

|Parameter                      |Component|Change     |Description                                                                                                                                                            |
|-------------------------------|---------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|ib_mad_timeout                 | AM      | Removed   |Maximum time [in milliseconds] to wait for MADs transaction to complete. The parameter is no longer displayed in the configuration file, help message, and README      |
|ib_mad_retries                 | AM      | Removed   |Maximum number of retries for timed out MADs transaction. The parameter is no longer displayed in the configuration file, help message, and README                     |
|smx_sock_addr_family           | SD      | New       |Determines which address family will be used in SMX's listening socket. The value needs to be one of the following: { ipv4, ipv6 }                                     |
|smx_sock_addr_family           | AM      | New       |Determines which address family will be used in SMX's listening socket. The value needs to be one of the following: { ipv4, ipv6 }                                     |
|span_all_agg_nodes             | AM      | New       |Generate trees that span all possible aggregation nodes. Relevant only if topology_type is tree.                                                                       |
|fabric_smdb_file               | AM      | New       |Topology information generated by SM (v4.9 or later)                                                                                                                   |
|hyper_cube_coordinates_file    | AM      | Deprecated|Default changed|Hyper Cube coordinates file. Replaced by fabric_smdb_file                                                                                              |
|root_guids_file                | AM      | Deprecated|Root guids file. Replaced by fabric_smdb_file                                                                                                                          |
|ib_dev                         | SD      | Removed   |IB port SD bind to                                                                                                                                                     |

[HEAD](https://github.com/Mellanox/sharp/commit/a1465daa061789c061fc4b87175443efac69434e)
[Full Changelog](https://github.com/Mellanox/sharp/compare/v1.2...6db56f19c11b0c972dfa720c1e3998f294905f39)

## v1.2

### Main features

#### Support for hypercube topology

- [#1480](https://github.com/Mellanox/sharp/pull/1480): sharp_am: Hypercube - support single AN subnets
- [#1457](https://github.com/Mellanox/sharp/pull/1457): Fix hc dim calc
- [#1399](https://github.com/Mellanox/sharp/pull/1399): sharp_am: TreeManager - support different dimensions order for HC trees
- [#1388](https://github.com/Mellanox/sharp/pull/1388): sharp_am: hypercube support

#### Topology update

- [#1495](https://github.com/Mellanox/sharp/pull/1495): Qp recovery
- [#1463](https://github.com/Mellanox/sharp/pull/1463): sharp_am: Support reloading of modified lst file.
- [#1418](https://github.com/Mellanox/sharp/pull/1418): Fabric update 1
- [#1431](https://github.com/Mellanox/sharp/pull/1431): fabric_graph.cpp: Fix uint8_t overflow issue

#### Integration with Job Scheduler

- [#1434](https://github.com/Mellanox/sharp/pull/1434): JS integration: Add managed mode flow to libsharp and SD, Add sharp_job_quota executable

#### CA ftree list transfer

- [#1362](https://github.com/Mellanox/sharp/pull/1362): Remove host name from ftree-ca sm_data msgs
- [#1355](https://github.com/Mellanox/sharp/pull/1355): SD: Add support for SM data (ftree-ca) transfer
- [#1352](https://github.com/Mellanox/sharp/pull/1352): Add SM data transfer support in API & libsharp
- [#1351](https://github.com/Mellanox/sharp/pull/1351): AM: Add support for transfer of SM data to MPI
- [#1349](https://github.com/Mellanox/sharp/pull/1349): FTree CA transfer feature - Add SMX msgs

#### Non-blocking API

- [#1372](https://github.com/Mellanox/sharp/pull/1372): sharp_coll: non blocking collective API

### Experimental features

- [#9299caa](https://github.com/Mellanox/sharp/commit/9299caa709524cf9db56b4c1f65dbacb692bd66b): Reorganize sharp_benchmark.sh. The script has been split to sharp_manager.sh, sharp_benchmark.sh and sharp_params.ini

### Internal changes

- [#1479](https://github.com/Mellanox/sharp/pull/1479): sharp option parser: Add mandatory args support
- [#1438](https://github.com/Mellanox/sharp/pull/1438): Change structure of the UFM deployment tarball
- [#1436](https://github.com/Mellanox/sharp/pull/1436): Fix rpath issues in case of running sharp_am from chroot. Applicable to UFM appliance.
- [#1419](https://github.com/Mellanox/sharp/pull/1419): Add ufm_sharp_unpack script for UFM team using
- [#1401](https://github.com/Mellanox/sharp/pull/1401): Build SHArP packages for UFM integration
- [#1346](https://github.com/Mellanox/sharp/pull/1346): sharpd/sharpd.c: Make job failure status more informative
- [#1345](https://github.com/Mellanox/sharp/pull/1345): sharpd/sharpd.c: Log configured IB device
- Integrate Valgrind into SHArP's Jenkins job

### Fixed

- [#1500](https://github.com/Mellanox/sharp/pull/1500): Fix incorrect return value of "git diff-index --quiet HEAD -- ./" by running "git status" before it
- [#1478](https://github.com/Mellanox/sharp/pull/1478): Add "backlog" to SMX socket configuration options
- [#1491](https://github.com/Mellanox/sharp/pull/1491): retry sharp_get_job_data_len only if return len is 0
- [#1460](https://github.com/Mellanox/sharp/pull/1460): Fix mcast attach failure
- [#1440](https://github.com/Mellanox/sharp/pull/1440): SMX: Fix timeouts when application messages itself
- [#1409](https://github.com/Mellanox/sharp/pull/1409): sharp_am: Fix AggNodeFabricInfo memory leak
- [#1402](https://github.com/Mellanox/sharp/pull/1402): sharp.c: Fix memory leak
- [#1398](https://github.com/Mellanox/sharp/pull/1398): sharp_coll: set target header only at rank 0
- [#1356](https://github.com/Mellanox/sharp/pull/1356): sharpd: Fix tree connection local port selection
- [#1358](https://github.com/Mellanox/sharp/pull/1358): sharpd/sharpd.c: Keep job running in case of multicast disabled in AM
- [#83fe027](https://github.com/Mellanox/sharp/commit/83fe02777c1bff088ec1f53c3fc877b476342a26): disable mcast on group if mgid == 0

### Added

- [#1496](https://github.com/Mellanox/sharp/pull/1496): sharp option parser: Add more usage information, Update default log function.
- [#1461](https://github.com/Mellanox/sharp/pull/1461): Add sharp_am_smx_address utility
- [#1383](https://github.com/Mellanox/sharp/pull/1383): AM: Add options for retrying to parse fabric.lst
- [#1391](https://github.com/Mellanox/sharp/pull/1391): stats: request finish transport counter
- [#1377](https://github.com/Mellanox/sharp/pull/1377): coll: Add a logging callback function for libsharp
- [#1365](https://github.com/Mellanox/sharp/pull/1365): client/sharp.c: Add error logging, time measures
- [#7c8d513](https://github.com/Mellanox/sharp/commit/7c8d51317e21774dbc24b2aede058d70343e084a): Add a tester for parallel group creation

### Changed

- [#1507](https://github.com/Mellanox/sharp/pull/1507): Readme and settings
- [#1442](https://github.com/Mellanox/sharp/pull/1442): disable group trim on group size 1.
- [#1406](https://github.com/Mellanox/sharp/pull/1406): Fix https://redmine.mellanox.com/issues/883764 (Prevent sharpd running on SM server)
- [#1387](https://github.com/Mellanox/sharp/pull/1387): SMX: Move socket configuration to AM/SD config
- [#1389](https://github.com/Mellanox/sharp/pull/1389): sharp_coll: soft preport recv buffers to UD endpoint
- [#1384](https://github.com/Mellanox/sharp/pull/1384): Replace proprietary logging in sharpd with alog
- [#a55862e](https://github.com/Mellanox/sharp/commit/a55862e23d973b809c682f774a95fb4c4d6c66f0): sharpd.c: Increase group allocation timeout up to 500 ms
- [#66362c8](https://github.com/Mellanox/sharp/commit/66362c8548c6cb526f899f95243425ca770dda0b): sharpd.c: Expose group allocation timeout in SD configuration

### Changes in parameters

#### HCOLL

Renames: SHARP_COLL_SHARP_ENABLE_MCAST_TARGET -> SHARP_COLL_ENABLE_MCAST_TARGET

#### AM and SD

New parameters:

|Parameter                     | Component| Description                                                                                        |
|------------------------------|----------|----------------------------------------------------------------------------------------------------|
|smx_sock_interface            | AM / SD  | "Network interface to be used by SMX. Default: empty string - Use first interface found in UP state|
|smx_sock_port                 | AM / SD  | The external port to be used by SMX. Default - 6126                                                |
|lst_file_timeout              | AM       | Length of timeout **in seconds** between attempts to load the LST file. Default - 3 seconds.       |
|lst_file_retries              | AM       | Max number of retry attempts when loading the LST file and encountering "_No such file_" errors. Default - 0 **meaning no retries**.|
|log_max_backup_files          | SD       | Number of backup log files. Used for log rotation                                                  |
|log_file_max_size             | SD       | Maximum size of a log file, in MBs. If value is 0, log rotation isn't used                         |
|mgmt_mode                     | SD       | When running in managed mode, SHArPD expects notifications from the Resource manager (Job scheduler). The possible values are: 0 - Unmanaged mode; 1 - Managed mode"|
|smx_sock_backlog              | SD       | Defines the maximum length to which the queue of pending connections for the SMX listen socket may grow|
|group_allocate_timeout        | SD       | Maximum time [in milliseconds] to wait for group allocation transaction to complete.               |

Removed:

Environment variables SMX_SOCK_PORT and SMX_SOCK_INTERFACE are removed.

### Installation

```
Usage: sharp_manager.sh <-o start/restart/stop/status/set/logs/ver> [-l hostlist] [-d hca:port] [-s am_server] [-f fabric_lst_file] [-r root_guids_file] [-h]
        -o - operation start/stop/status/set/logs/ver
                'start/restart/stop/status' - apply to SHArP daemons
                'set'  - set SHArP daemons runtime parameters
                'logs' - collect log from SHArP daemons
                'ver'  - show version of SHArP daemons
        -l - Compute nodes host list
        -d - device (hca:port)
        -s - AM server hostname
        -f - Path to fabric_lst_file
        -r - Path to root_guids_file
        -h - This help
```

- All local settings will be saved into ~/.sharp_local.ini
- On start sharp_manager.sh will create sharpd.cfg and sharp_am.cfg and will save their prev. vers.
- sharp_benchmark.sh behavior: for daemons startup use "-s", otherwise tests only
- sharp_setup_wizard.sh will provide comfortable way to set parameters

### Full log

- [v1.2.0](https://github.com/Mellanox/sharp/commit/9b685d98cdbf7d6b17d4c8002b7b14ab09f443c3)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.1.1..v1.2.0)

## v1.1

### Main features

- UD Multicast
- [Group multi-channel](https://github.com/Mellanox/sharp/wiki/Multi-channel-group)
- [Group trim](https://github.com/Mellanox/sharp/wiki/Group-trim)
- Limited support for SM handover/failover
- [Manual configuration for mapping between hosts and ANs](https://github.com/Mellanox/sharp/wiki/Support-for-mixed-fabric)
- [Error flow](https://github.com/Mellanox/sharp/wiki/Error-flow)

### Changes in parameters

#### HCOLL
New parameters:


| Name                                        | Description                                                                      |
|---------------------------------------------|----------------------------------------------------------------------------------|
|SHARP_COLL_ERROR_CHECK_INTERVAL              | Interval, in milli second, indicates the time between the error checks           |
|SHARP_COLL_ENABLE_GROUP_TRIM                 | Enable sharp group trim feature                                                  |
|SHARP_COLL_GROUP_IS_TARGET                   | Option to set group target for results                                           |
|SHARP_COLL_MCAST_TARGET_GROUP_SIZE_THRESHOLD | Group size threshold to enable mcast target                                      |
|SHARP_COLL_POLL_BATCH                        | How many CQ completions to poll on at once. Maximum:16                           |

#### AM
Removed:

| Name                                        | Description                                                                      |
|---------------------------------------------|----------------------------------------------------------------------------------|
| ibis_log_file                               | ibis log is redirected to AM log                                                 |
| ibis_log_size                               | ibis log is redirected to AM log                                                 |
| ibis_accum_log                              | ibis log is redirected to AM log                                                 |

New parameters:

| Name                                        | Description                                                                      |
|---------------------------------------------|----------------------------------------------------------------------------------|
| support_multicast                           | Enables UD MCAST support in AM. Default - "TRUE"                                 |
| sharp_multicast_signature                   | The signature that will be used in multicast group MGID                          |

Changed:

The new default for fabric_lst_file is opensm default : /var/log/opensm-subnet.lst

### Changes in APIs
- libsharp_coll:
    - New function is added: sharp_coll_group_trim .
- libsharp:
    - Error callback is removed.
    - New function is added: sharp_get_errors
    - Prototype changed: sharp_create_job, sharp_get_tree_info, sharp_get_tree_connect_info

### Changes in internal messages (SMX)
- New messages:
    - sharp_job_error
    - sharp_am_signal

- Removed messages:
    - sharp_get_job_data

- Changed messages:
    - sharp_job_data
        - QP config options is added
        - List of ANs is added
        - Hostnames are removed
        - List of qp indexes are added
    - sharp_begin_job
        - Enable MCAST parameter is added
        - Number of qp indexes is added


### Internal changes
- [#1132](https://github.com/Mellanox/sharp/pull/1132): sharp_opt_parser: support hiding parameters
- [#1137](https://github.com/Mellanox/sharp/pull/1137): FabricProvider - redirect IBIS logs
- [#1177](https://github.com/Mellanox/sharp/pull/1177): Add ib_qpc options to job_data message.
  Transfer QP creation paramters from AM to SD.

### Fixed
- [#1144](https://github.com/Mellanox/sharp/pull/1144): Fix memory corruption in SD.
- [#1226](https://github.com/Mellanox/sharp/pull/1226): Fix libsharp dependency
- [#1241](https://github.com/Mellanox/sharp/pull/1241): Fix error during service start/stop of sharpd/sharp_am when conf file doesn't contain pid_file param
- [#1233](https://github.com/Mellanox/sharp/pull/1233): sharp.spec.in: Fix sharp_daemons_setup.sh path
- [#1332](https://github.com/Mellanox/sharp/pull/1332): Fix mode of SHArP log files (sharp_am.log/sharpd.log)
- [#1327](https://github.com/Mellanox/sharp/pull/1327): bug fix: memory corruption in group trim response header unpack.
- [#1193](https://github.com/Mellanox/sharp/pull/1193): sharp_coll: check oob collective return value
- [#1174](https://github.com/Mellanox/sharp/pull/1174): sharp_benchmark.sh: Preload libsharp_coll.so
- [#1169](https://github.com/Mellanox/sharp/pull/1169): sharp_coll: fix error message
- [#1154](https://github.com/Mellanox/sharp/pull/1154): Try to create dirs for PIDFILE and LOCKFILE in any case.
  Fix PID and LOCK file creation.
- [#1142](https://github.com/Mellanox/sharp/pull/1142): sharp_coll: add job ID to log message
- [#1141](https://github.com/Mellanox/sharp/pull/1141): sharpd.c: Remove upper limit for log_level parameter
- [#1108](ps://github.com/Mellanox/sharp/pull/1108): sharp_coll: Fix big endian data hdr packing
- [#1128](https://github.com/Mellanox/sharp/pull/1128): sharpd.c: In sharpd_svr_receive when handling SHARPD_OP_GET_JOB_DATA only free job data message when smx_msg_pack succeeds

### Added
- [#1259](https://github.com/Mellanox/sharp/pull/1259): Test application for communicator trim
- [#1287](https://github.com/Mellanox/sharp/pull/1287): test/Makefile.am: Add ib_dev to SD configuration file
- [#1320](https://github.com/Mellanox/sharp/pull/1320): Add SHArP parameters usually used in testing to sharp_benchmark.sh
- [#1309](https://github.com/Mellanox/sharp/pull/1309): Split sharp_benchmark.sh to "start up only" and "tests only" parts
- [#1315](https://github.com/Mellanox/sharp/pull/1315): sharp_coll: Add option for poll batch size.
- [#1197](https://github.com/Mellanox/sharp/pull/1197): Use sharp_opt_parser for SD option configuration
- [#1175](https://github.com/Mellanox/sharp/pull/1175): libsharp: Error flow API modifications
- [#1146](https://github.com/Mellanox/sharp/pull/1146): Add capability to start-up scripts to transefer any env. vars to SHArP daemons
OB

### Changed

- [#1256](https://github.com/Mellanox/sharp/pull/1256): sharp_am: Exit if LoadFabric failed.
- [#1194](https://github.com/Mellanox/sharp/pull/1194): sharpd: Remove AggMgrMode option
- [#1186](https://github.com/Mellanox/sharp/pull/1186): sharp.spec.in: Do not install sharpd and sharp_am by default
- [#1187](https://github.com/Mellanox/sharp/pull/1187): sharp_am: Sort host info list in job_data
- [#1179](https://github.com/Mellanox/sharp/pull/1179): SMX refactoring: Remove get_job_data msg, remove fields from job_data.
  Remove obsolete SMX message (sharp_get_job_data) and fields (max_tree_radix_in_fabric and radix).
- [#1168](https://github.com/Mellanox/sharp/pull/1168): SMX: replace hostnames with sd_info in job data msg.
  Remove hostnames from job data.
- [#1153](https://github.com/Mellanox/sharp/pull/1153): sharp_coll: improve sharp coll API.
  Add C++ interoperability.
- [#1113](https://github.com/Mellanox/sharp/pull/1113): Redefine conf files default name and location

## Unreleased

### Full log
- [HEAD](https://github.com/Mellanox/sharp/commit/v1.1.0)
- [Full Changelog](https://github.com/Mellanox/sharp/compare/v1.0.0...v1.1.0)
