|
1
|
ChangeLog |
|
2
|
|
|
3
|
|
|
4
|
English version [by Google](https: |
|
5
|
and [by Yandex](https: |
|
6
|
|
|
7
|
## v0.12.4 (Арта-333) от 2023-03-03 |
|
8
|
|
|
9
|
Стабилизирующий выпуск с исправлением обнаруженных ошибок, устранением |
|
10
|
недочетов и технических долгов. Ветка 0.12 считается готовой к |
|
11
|
продуктовому использованию, получает статус стабильной и далее будет |
|
12
|
получать только исправление ошибок. Разработка будет продолжена в ветке |
|
13
|
0.13, а ветка 0.11 становится архивной. |
|
14
|
|
|
15
|
``` |
|
16
|
63 files changed, 1161 insertions(+), 569 deletions(-) |
|
17
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
18
|
``` |
|
19
|
|
|
20
|
Благодарности: |
|
21
|
|
|
22
|
- Max <maxc0d3r@protonmail.com> за сообщение о проблеме ERROR_SHARING_VIOLATION |
|
23
|
в режиме MDBX_EXCLUSIVE на Windows. |
|
24
|
- Alisher Ashyrov <https: |
|
25
|
с assert-проверкой и содействие в отладке. |
|
26
|
- Masatoshi Fukunaga <https: |
|
27
|
`put(MDBX_UPSERT+MDBX_ALLDUPS)` для случая замены всех значений в subDb. |
|
28
|
|
|
29
|
Исправления (без корректировок новых функций): |
|
30
|
|
|
31
|
- Устранен регресс после коммита 474391c83c5f81def6fdf3b0b6f5716a87b78fbf, |
|
32
|
приводящий к возврату ERROR_SHARING_VIOLATION в Windows при открытии БД |
|
33
|
в режиме MDBX_EXCLUSIVE для чтения-записи. |
|
34
|
|
|
35
|
- Добавлено ограничение размера отображения при коротком read-only файле, для |
|
36
|
предотвращения ошибки ERROR_NOT_ENOUGH_MEMORY в Windows, которая возникает |
|
37
|
в этом случае и совсем не информативна для пользователя. |
|
38
|
|
|
39
|
- Произведен рефакторинг `dxb_resize()`, в том числе, для устранения срабатывания |
|
40
|
assert-проверки `size_bytes == env->me_dxb_mmap.current` в специфических |
|
41
|
многопоточных сценариях использования. Проверка срабатывала только в |
|
42
|
отладочных сборках, при специфическом наложении во времени читающей и |
|
43
|
пишущей транзакции в разных потоках, одновременно с изменением размера БД. |
|
44
|
Кроме срабатывание проверки, каких-либо других последствий не возникало. |
|
45
|
|
|
46
|
- Устранена проблема в `put(MDBX_UPSERT+MDBX_ALLDUPS)` для случая замены |
|
47
|
всех значений единственного ключа в subDb. В ходе этой операции subDb |
|
48
|
становится полностью пустой, без каких-либо страниц и именно эта |
|
49
|
ситуация не была учтена в коде, что приводило к повреждению БД |
|
50
|
при фиксации такой транзакции. |
|
51
|
|
|
52
|
- Устранена излишняя assert-проверка внутри `override_meta()`. |
|
53
|
Что в отладочных сборках могло приводить к ложным срабатываниям |
|
54
|
при восстановлении БД, в том числе при автоматическом откате слабых |
|
55
|
мета-страниц. |
|
56
|
|
|
57
|
- Скорректированы макросы `__cold`/`__hot`, в том числе для устранения проблемы |
|
58
|
`error: inlining failed in call to ‘always_inline FOO(...)’: target specific option mismatch` |
|
59
|
при сборке посредством GCC >10.x для SH4. |
|
60
|
|
|
61
|
Ликвидация технических долгов и мелочи: |
|
62
|
|
|
63
|
- Исправлены многочисленные опечатки в документации. |
|
64
|
- Доработан тест для полной стохастической проверки `MDBX_EKEYMISMATCH` в режиме `MDBX_APPEND`. |
|
65
|
- Расширены сценарии запуска `mdbx_chk` из CMake-тестов для проверки как в обычном, |
|
66
|
так и эксклюзивном режимах чтения-записи. |
|
67
|
- Уточнены спецификаторы `const` и `noexcept` для нескольких методов в C++ API. |
|
68
|
- Устранено использование стека под буферы для `wchar`-преобразования путей. |
|
69
|
- Для Windows добавлена функция `mdbx_env_get_path()` для получения пути к БД |
|
70
|
в формате многобайтных символов. |
|
71
|
- Добавлены doxygen-описания для API с широкими символами. |
|
72
|
- Устранены предупреждения статического анализатора MSVC, |
|
73
|
все они были несущественные, либо ложные. |
|
74
|
- Устранено ложное предупреждение GCC при сборке для SH4. |
|
75
|
- Добавлена поддержка ASAN (Address Sanitizer) при сборке посредством MSVC. |
|
76
|
- Расширен набор перебираемых режимов в скрипте `test/long_stochastic.sh`, |
|
77
|
добавлена опция `--extra`. |
|
78
|
- В C++ API добавлена поддержка расширенных опций времени выполнения `mdbx::extra_runtime_option`, |
|
79
|
аналогично `enum MDBX_option_t` из C API. |
|
80
|
- Вывод всех счетчиков page-operations в `mdbx_stat`. |
|
81
|
|
|
82
|
|
|
83
|
|
|
84
|
|
|
85
|
|
|
86
|
## v0.12.3 (Акула) от 2023-01-07 |
|
87
|
|
|
88
|
Выпуск с существенными доработками и новой функциональностью в память о закрытом open-source |
|
89
|
[проекте "Акула"](https: |
|
90
|
|
|
91
|
Добавлена prefault-запись, переделан контроль “некогерентности” unified page/buffer cache, изменена тактика слияния страниц и т.д. |
|
92
|
Стало ещё быстрее, в некоторых сценариях вдвое. |
|
93
|
|
|
94
|
``` |
|
95
|
20 files changed, 4508 insertions(+), 2928 deletions(-) |
|
96
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
97
|
``` |
|
98
|
|
|
99
|
Благодарности: |
|
100
|
|
|
101
|
- [Alex Sharov](https: |
|
102
|
- [Simon Leier](https: |
|
103
|
|
|
104
|
Новое: |
|
105
|
|
|
106
|
- Использование адреса [https: |
|
107
|
для отсылки к сохранённым в web.archive.org копиям ресурсов, уничтоженных администрацией Github. |
|
108
|
|
|
109
|
- Реализована prefault-запись при выделении страниц для read-write отображений. |
|
110
|
Это приводит к кратному снижению системных издержек и существенному увеличению |
|
111
|
производительности в соответствующих сценариях использования, когда: |
|
112
|
- размер БД и объём данных существенно больше ОЗУ; |
|
113
|
- используется режим `MDBX_WRITEMAP`; |
|
114
|
- не-мелкие транзакции (по ходу транзакции выделяется многие сотни или тысячи страниц). |
|
115
|
|
|
116
|
В режиме `MDBX_WRITEMAP` выделение/переиспользование страниц приводит |
|
117
|
к page-fault и чтению страницы с диска, даже если содержимое страницы |
|
118
|
не нужно (будет перезаписано). Это является следствием работы подсистемы |
|
119
|
виртуальной памяти, а штатный способ лечения через `MADV_REMOVE` |
|
120
|
работает не на всех ФС и обычно дороже получаемой экономии. |
|
121
|
|
|
122
|
Теперь в libmdbx используется "упреждающая запись" таких страниц, |
|
123
|
которая на системах с [unified page cache](https: |
|
124
|
приводит к "вталкиванию" данных, устраняя необходимость чтения с диска при |
|
125
|
обращении к такой странице памяти. |
|
126
|
|
|
127
|
Новый функционал работает в согласованности с автоматическим управлением read-ahead |
|
128
|
и кэшем статуса присутствия страниц в ОЗУ, посредством [mincore()](https: |
|
129
|
|
|
130
|
- Добавлена опция `MDBX_opt_prefault_write_enable` для возможности принудительного |
|
131
|
включения/выключения prefault-записи. |
|
132
|
|
|
133
|
- Реализован динамический выбор между сквозной записью на диск и обычной записью |
|
134
|
с последующим [fdatasync()](https: |
|
135
|
управляемый опцией `MDBX_opt_writethrough_threshold`. |
|
136
|
|
|
137
|
В долговечных (durable) режимах данные на диск могут быть сброшены двумя способами: |
|
138
|
- сквозной записью через файловый дескриптор открытый с `O_DSYNC`; |
|
139
|
- обычной записью с последующим вызовом `fdatasync()`. |
|
140
|
|
|
141
|
Первый способ выгоднее при записи малого количества страниц и/или если |
|
142
|
канал взаимодействия с диском/носителем имеет близкую к нулю задержку. |
|
143
|
Второй способ выгоднее если требуется записать много страниц и/или канал |
|
144
|
взаимодействия имеет весомую задержку (датацентры, облака). Добавленная |
|
145
|
опция `MDBX_opt_writethrough_threshold` позволяет во время выполнения |
|
146
|
задать порог для динамического выбора способа записи в зависимости от |
|
147
|
объема и конкретных условия использования. |
|
148
|
|
|
149
|
- Автоматическая установка `MDBX_opt_rp_augment_limit` в зависимости от размера БД. |
|
150
|
|
|
151
|
- Запрещение разного режима `MDBX_WRITEMAP` между процессами в режимах |
|
152
|
с отложенной/ленивой записью, так как в этом случае невозможно |
|
153
|
обеспечить сброс данных на диск во всех случаях на всех поддерживаемых платформах. |
|
154
|
|
|
155
|
- Добавлена опция сборки `MDBX_MMAP_USE_MS_ASYNC` позволяющая отключить |
|
156
|
использование системного вызова `msync(MS_ASYNC)`, в использовании |
|
157
|
которого нет необходимости на подавляющем большинстве актуальных ОС. |
|
158
|
По-умолчанию `MDBX_MMAP_USE_MS_ASYNC=0` (выключено) на Linux и других |
|
159
|
системах с unified page cache. Такое поведение (без использования |
|
160
|
`msync(MS_ASYNC)`) соответствует неизменяемой (hardcoded) логике LMDB. В |
|
161
|
результате, в простых/наивных бенчмарках, libmdbx опережает LMDB |
|
162
|
примерно также как при реальном применении. |
|
163
|
|
|
164
|
На всякий случай стоит еще раз отметить/напомнить, что на Windows |
|
165
|
предположительно libmdbx будет отставать от LMDB в сценариях с |
|
166
|
множеством мелких транзакций, так как libmdbx осознанно использует на |
|
167
|
Windows файловые блокировки, которые медленные (плохо реализованы в ядре |
|
168
|
ОС), но позволяют застраховать пользователей от массы неверных действий |
|
169
|
приводящих к повреждению БД. |
|
170
|
|
|
171
|
- Поддержка не-печатных имен для subDb. |
|
172
|
|
|
173
|
- Добавлен явный выбор `tls_model("local-dynamic")` для обхода проблемы |
|
174
|
`relocation R_X86_64_TPOFF32 against FOO cannot be used with -shared` |
|
175
|
из-за ошибки в CLANG приводящей к использованию неверного режима `ls_model`. |
|
176
|
|
|
177
|
- Изменение тактики слияния страниц при удалении. |
|
178
|
Теперь слияние выполняется преимущественно с уже измененной/грязной страницей. |
|
179
|
Если же справа и слева обе страницы с одинаковым статусом, |
|
180
|
то с наименее заполненной, как прежде. В сценариях с массивным удалением |
|
181
|
это позволяет увеличить производительность до 50%. |
|
182
|
|
|
183
|
- Добавлен контроль отсутствия LCK-файлов с альтернативным именованием. |
|
184
|
|
|
185
|
Исправления (без корректировок новых функций): |
|
186
|
|
|
187
|
- Изменение размера отображения если это требуется для сброса данных на |
|
188
|
диск при вызове `mdbx_env_sync()` из параллельного потока выполнения вне |
|
189
|
работающей транзакции. |
|
190
|
|
|
191
|
- Исправление регресса после коммита db72763de049d6e4546f838277fe83b9081ad1de от 2022-10-08 |
|
192
|
в логике возврата грязных страниц в режиме `MDBX_WRITEMAP`, из-за чего |
|
193
|
освободившиеся страницы использовались не немедленно, а попадали в |
|
194
|
retired-список совершаемой транзакции и происходил необоснованный рост |
|
195
|
размера транзакции. |
|
196
|
|
|
197
|
- Устранение SIGSEGV или ошибочного вызова `free()` в ситуациях |
|
198
|
повторного открытия среды посредством `mdbx_env_open()`. |
|
199
|
|
|
200
|
- Устранение ошибки совершенной в коммите fe20de136c22ed3bc4c6d3f673e79c106e824f60 от 2022-09-18, |
|
201
|
в результате чего на Linux в режиме `MDBX_WRITEMAP` никогда не вызывался `msync()`. |
|
202
|
Проблема существует только в релизе 0.12.2. |
|
203
|
|
|
204
|
- Добавление подсчета грязных страниц в `MDBX_WRITEMAP` для предоставления посредством `mdbx_txn_info()` |
|
205
|
актуальной информации об объеме изменений в процессе транзакций чтения-записи. |
|
206
|
|
|
207
|
- Исправление несущественной опечатки в условиях `#if` определения порядка байт. |
|
208
|
|
|
209
|
- Исправление сборки для случая `MDBX_PNL_ASCENDING=1`. |
|
210
|
|
|
211
|
Ликвидация технических долгов и мелочи: |
|
212
|
|
|
213
|
- Доработка поддержки авто-слияния записей GC внутри `page_alloc_slowpath()`. |
|
214
|
- Устранение несущественных предупреждений Coverity. |
|
215
|
- Использование единого курсора для поиска в GC. |
|
216
|
- Переработка внутренних флагов связанных с выделением страниц из GC. |
|
217
|
- Доработка подготовки резерва перед обновлением GC при включенном BigFoot. |
|
218
|
- Оптимизация `pnl_merge()` для случаев неперекрывающихся объединяемых списков. |
|
219
|
- Оптимизация поддержки отсортированного списка страниц в `dpl_append()`. |
|
220
|
- Ускорение работы `mdbx_chk` при обработке пользовательских записей в `@MAIN`. |
|
221
|
- Переработка LRU-отметок для спиллинга. |
|
222
|
- Переработка контроля "некогерентности" Unified page cache для уменьшения накладных расходов. |
|
223
|
- Рефакторинг и микрооптимизация. |
|
224
|
|
|
225
|
|
|
226
|
|
|
227
|
|
|
228
|
|
|
229
|
## v0.12.2 (Иван Ярыгин) от 2022-11-11 |
|
230
|
|
|
231
|
Выпуск с существенными доработками и новой функциональностью |
|
232
|
в память о российском борце [Иване Сергеевиче Ярыгине](https: |
|
233
|
|
|
234
|
На Олимпийских играх в Мюнхене в 1972 году Иван Ярыгин уложил всех соперников на лопатки, |
|
235
|
суммарно затратив менее 9 минут. Этот рекорд никем не побит до сих пор. |
|
236
|
|
|
237
|
``` |
|
238
|
64 files changed, 5573 insertions(+), 2510 deletions(-) |
|
239
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
240
|
``` |
|
241
|
|
|
242
|
Новое: |
|
243
|
|
|
244
|
- Поддержка всех основных опций при сборке посредством CMake. |
|
245
|
|
|
246
|
- Требования к CMake понижены до версии 3.0.2 для возможности сборки для устаревших платформ. |
|
247
|
|
|
248
|
- Добавлена возможность профилирования работы GC в сложных и/или нагруженных |
|
249
|
сценариях (например Ethereum/Erigon). По-умолчанию соответствующий код отключен, |
|
250
|
а для его активации необходимо указать опцию сборки `MDBX_ENABLE_PROFGC=1`. |
|
251
|
|
|
252
|
- Добавлена функция `mdbx_env_warmup()` для "прогрева" БД с возможностью |
|
253
|
закрепления страниц в памяти. |
|
254
|
В утилиты `mdbx_chk`, `mdbx_copy` и `mdbx_dump` добавлены опции `-u` и `-U` |
|
255
|
для активации соответствующего функционала. |
|
256
|
|
|
257
|
- Отключение учета «грязных» страниц в не требующих этого режимах |
|
258
|
(`MDBX_WRITEMAP` при `MDBX_AVOID_MSYNC=0`). Доработка позволяет снизить |
|
259
|
накладные расходы и была запланирована давно, но откладывалась так как |
|
260
|
требовала других изменений. |
|
261
|
|
|
262
|
- Вытеснение из памяти (спиллинг) «грязных» страниц с учетом размера |
|
263
|
large/overflow-страниц. Доработка позволяет корректно соблюдать политику |
|
264
|
задаваемую опциями `MDBX_opt_txn_dp_limit`, |
|
265
|
`MDBX_opt_spill_max_denominator`, `MDBX_opt_spill_min_denominator` и |
|
266
|
была запланирована давно, но откладывалась так как требовала других |
|
267
|
изменений. |
|
268
|
|
|
269
|
- Для Windows в API добавлены UNICODE-зависимые определения макросов |
|
270
|
`MDBX_DATANAME`, `MDBX_LOCKNAME` и `MDBX_LOCK_SUFFIX`. |
|
271
|
|
|
272
|
- Переход на преимущественное использование типа `size_t` для |
|
273
|
уменьшения накладных расходов на платформе Эльбрус. |
|
274
|
|
|
275
|
- В API добавлены функции `mdbx_limits_valsize4page_max()` и |
|
276
|
`mdbx_env_get_valsize4page_max()` возвращающие максимальный размер в |
|
277
|
байтах значения, которое может быть размещена в одной |
|
278
|
large/overflow-странице, а не последовательности из двух или более таких |
|
279
|
страниц. Для таблиц с поддержкой дубликатов вынос значений на |
|
280
|
large/overflow-страницы не поддерживается, поэтому результат совпадает с |
|
281
|
`mdbx_limits_valsize_max()`. |
|
282
|
|
|
283
|
- В API добавлены функции `mdbx_limits_pairsize4page_max()`и |
|
284
|
`mdbx_env_get_pairsize4page_max()` возвращающие в байтах максимальный |
|
285
|
суммарный размер пары ключ-значение для их размещения на одной листовой |
|
286
|
страницы, без выноса значения на отдельную large/overflow-страницу. Для |
|
287
|
таблиц с поддержкой дубликатов вынос значений на large/overflow-страницы |
|
288
|
не поддерживается, поэтому результат определяет максимальный/допустимый |
|
289
|
суммарный размер пары ключ-значение. |
|
290
|
|
|
291
|
- Реализовано использование асинхронной (overlapped) записи в Windows, |
|
292
|
включая использования небуфферизированного ввода-вывода и `WriteGather()`. |
|
293
|
Это позволяет сократить накладные расходы и частично обойти проблемы |
|
294
|
Windows с низкой производительностью ввода-вывода, включая большие |
|
295
|
задержки `FlushFileBuffers()`. Новый код также обеспечивает консолидацию |
|
296
|
записываемых регионов на всех платформах, а на Windows использование |
|
297
|
событий (events) сведено к минимум, одновременно с автоматических |
|
298
|
использованием `WriteGather()`. Поэтому ожидается существенное снижение |
|
299
|
накладных расходов взаимодействия с ОС, а в Windows это ускорение, в |
|
300
|
некоторых сценариях, может быть кратным в сравнении с LMDB. |
|
301
|
|
|
302
|
- Добавлена опция сборки `MDBX_AVOID_MSYNC`, которая определяет |
|
303
|
поведение libmdbx в режиме `MDBX_WRITE_MAP` (когда данные изменяются |
|
304
|
непосредственно в отображенных в ОЗУ страницах БД): |
|
305
|
|
|
306
|
* Если `MDBX_AVOID_MSYNC=0` (по умолчанию на всех системах кроме Windows), |
|
307
|
то (как прежде) сохранение данных выполняется посредством `msync()`, |
|
308
|
либо `FlushViewOfFile()` на Windows. На платформах с полноценной |
|
309
|
подсистемой виртуальной памяти и адекватным файловым вводом-выводом |
|
310
|
это обеспечивает минимум накладных расходов (один системный вызов) |
|
311
|
и максимальную производительность. Однако, на Windows приводит |
|
312
|
к значительной деградации, в том числе из-за того что после |
|
313
|
`FlushViewOfFile()` требуется также вызов `FlushFileBuffers()` |
|
314
|
с массой проблем и суеты внутри ядра ОС. |
|
315
|
|
|
316
|
* Если `MDBX_AVOID_MSYNC=1` (по умолчанию только на Windows), то |
|
317
|
сохранение данных выполняется явной записью в файл каждой измененной |
|
318
|
страницы БД. Это требует дополнительных накладных расходов, как |
|
319
|
на отслеживание измененных страниц (ведение списков "грязных" |
|
320
|
страниц), так и на системные вызовы для их записи. |
|
321
|
Кроме этого, с точки зрения подсистемы виртуальной памяти ядра ОС, |
|
322
|
страницы БД измененные в ОЗУ и явно записанные в файл, могут либо |
|
323
|
оставаться "грязными" и быть повторно записаны ядром ОС позже, |
|
324
|
либо требовать дополнительных накладных расходов для отслеживания |
|
325
|
PTE (Page Table Entries), их модификации и дополнительного копирования |
|
326
|
данных. Тем не менее, по имеющейся информации, на Windows такой путь |
|
327
|
записи данных в целом обеспечивает более высокую производительность. |
|
328
|
|
|
329
|
- Улучшение эвристики включения авто-слияния записей GC. |
|
330
|
|
|
331
|
- Изменение формата LCK и семантики некоторых внутренних полей. Версии |
|
332
|
libmdbx использующие разный формат не смогут работать с одной БД |
|
333
|
одновременно, а только поочередно (LCK-файл переписывается при открытии |
|
334
|
первым открывающим БД процессом). |
|
335
|
|
|
336
|
- В `C++` API добавлены методы фиксации транзакции с получением информации |
|
337
|
о задержках. |
|
338
|
|
|
339
|
- Added `MDBX_HAVE_BUILT IN_CPU_SUPPORTS` build option to control use GCC's |
|
340
|
`__builtin_cpu_supports()` function, which could be unavailable on a fake |
|
341
|
OSes (macos, ios, android, etc). |
|
342
|
|
|
343
|
Исправления (без корректировок вышеперечисленных новых функций): |
|
344
|
|
|
345
|
- Устранения ряда предупреждений при сборке посредством MinGW. |
|
346
|
- Устранение ложно-положительных сообщений от Valgrind об использовании |
|
347
|
не инициализированных данных из-за выравнивающих зазоров в `struct troika`. |
|
348
|
- Исправлен возврат неожиданной ошибки `MDBX_BUSY` из функций `mdbx_env_set_option()`, |
|
349
|
`mdbx_env_set_syncbytes()` и `mdbx_env_set_syncperiod()`. |
|
350
|
- Небольшие исправления для совместимости с CMake 3.8 |
|
351
|
- Больше контроля и осторожности (паранойи) для страховки от дефектов `mremap()`. |
|
352
|
- Костыль для починки сборки со старыми версиями `stdatomic.h` из GNU Lib C, |
|
353
|
где макросы `ATOMIC_*_LOCK_FREE` ошибочно переопределяются через функции. |
|
354
|
- Использование `fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64)` при наличии. |
|
355
|
Это решает проблему срабатывания проверочного утверждения при сборке для |
|
356
|
платформ где тип `off_t` шире соответствующих полей `структуры flock`, |
|
357
|
используемой для блокировки файлов. |
|
358
|
- Доработан сбор информации о задержках при фиксации транзакций: |
|
359
|
* Устранено искажение замеров длительности обновления GC |
|
360
|
при включении отладочного внутреннего аудита; |
|
361
|
* Защита от undeflow-нуля только общей задержки в метриках, |
|
362
|
чтобы исключить ситуации, когда сумма отдельных стадий |
|
363
|
больше общей длительности. |
|
364
|
- Ряд исправлений для устранения срабатываний проверочных утверждения в |
|
365
|
отладочных сборках. |
|
366
|
- Более осторожное преобразование к типу `mdbx_tid_t` для устранения |
|
367
|
предупреждений. |
|
368
|
- Исправление лишнего сброса данных на диск в режиме `MDBX_SAFE_NOSYNC` |
|
369
|
при обновлении GC. |
|
370
|
- Fixed an extra check for `MDBX_APPENDDUP` inside `mdbx_cursor_put()` |
|
371
|
which could result in returning `MDBX_EKEYMISMATCH` for valid cases. |
|
372
|
- Fixed nasty `clz()` bug (by using `_BitScanReverse()`, only MSVC builds affected). |
|
373
|
|
|
374
|
Мелочи: |
|
375
|
|
|
376
|
- Исторические ссылки cвязанные с удалённым на ~~github~~ проектом перенаправлены на [web.archive.org](https://web.archive.org/web/https://github.com/erthink/libmdbx). |
|
377
|
- Синхронизированны конструкции CMake между проектами. |
|
378
|
- Добавлено предупреждение о небезопасности RISC-V. |
|
379
|
- Добавлено описание параметров `MDBX_debug_func` и `MDBX_debug_func`. |
|
380
|
- Добавлено обходное решение для минимизации ложно-положительных |
|
381
|
конфликтов при использовании файловых блокировок в Windows. |
|
382
|
- Проверка атомарности C11-операций c 32/64-битными данными. |
|
383
|
- Уменьшение в 42 раза значения по-умолчанию для `me_options.dp_limit` |
|
384
|
в отладочных сборках. |
|
385
|
- Добавление платформы `gcc-riscv64-linux-gnu` в список для цели `cross-gcc`. |
|
386
|
- Небольшие правки скрипта `long_stochastic.sh` для работы в Windows. |
|
387
|
- Удаление ненужного вызова `LockFileEx()` внутри `mdbx_env_copy()`. |
|
388
|
- Добавлено описание использования файловых дескрипторов в различных режимах. |
|
389
|
- Добавлено использование `_CrtDbgReport()` в отладочных сборках. |
|
390
|
- Fixed an extra ensure/assertion check of `oldest_reader` inside `txn_end()`. |
|
391
|
- Removed description of deprecated usage of `MDBX_NODUPDATA`. |
|
392
|
- Fixed regression ASAN/Valgring-enabled builds. |
|
393
|
- Fixed minor MingGW warning. |
|
394
|
|
|
395
|
|
|
396
|
|
|
397
|
|
|
398
|
|
|
399
|
## v0.12.1 (Positive Proxima) at 2022-08-24 |
|
400
|
|
|
401
|
The planned frontward release with new superior features on the day of 20 anniversary of [Positive Technologies](https: |
|
402
|
|
|
403
|
``` |
|
404
|
37 files changed, 7604 insertions(+), 7417 deletions(-) |
|
405
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
406
|
``` |
|
407
|
|
|
408
|
New: |
|
409
|
|
|
410
|
- The `Big Foot` feature which significantly reduces GC overhead for processing large lists of retired pages from huge transactions. |
|
411
|
Now _libmdbx_ avoid creating large chunks of PNLs (page number lists) which required a long sequences of free pages, aka large/overflow pages. |
|
412
|
Thus avoiding searching, allocating and storing such sequences inside GC. |
|
413
|
- Improved hot/online validation and checking of database pages both for more robustness and performance. |
|
414
|
- New solid and fast method to latch meta-pages called `Troika`. |
|
415
|
The minimum of memory barriers, reads, comparisons and conditional transitions are used. |
|
416
|
- New `MDBX_VALIDATION` environment options to extra validation of DB structure and pages content for carefully/safe handling damaged or untrusted DB. |
|
417
|
- Accelerated ×16/×8/×4 by AVX512/AVX2/SSE2/Neon implementations of search page sequences. |
|
418
|
- Added the `gcrtime_seconds16dot16` counter to the "Page Operation Statistics" that accumulates time spent for GC searching and reclaiming. |
|
419
|
- Copy-with-compactification now clears/zeroes unused gaps inside database pages. |
|
420
|
- The `C` and `C++` APIs has been extended and/or refined to simplify using `wchar_t` pathnames. |
|
421
|
On Windows the `mdbx_env_openW()`, ``mdbx_env_get_pathW()`()`, `mdbx_env_copyW()`, `mdbx_env_open_for_recoveryW()` are available for now, |
|
422
|
but the `mdbx_env_get_path()` has been replaced in favor of `mdbx_env_get_pathW()`. |
|
423
|
- Added explicit error message for Buildroot's Microblaze toolchain maintainers. |
|
424
|
- Added `MDBX_MANAGE_BUILD_FLAGS` build options for CMake. |
|
425
|
- Speed-up internal `bsearch`/`lower_bound` implementation using branchless tactic, including workaround for CLANG x86 optimiser bug. |
|
426
|
- A lot internal refinement and micro-optimisations. |
|
427
|
- Internally counted volume of dirty pages (unused for now but for coming features). |
|
428
|
|
|
429
|
Fixes: |
|
430
|
|
|
431
|
- Never use modern `__cxa_thread_atexit()` on Apple's OSes. |
|
432
|
- Don't check owner for finished transactions. |
|
433
|
- Fixed typo in `MDBX_EINVAL` which breaks MingGW builds with CLANG. |
|
434
|
|
|
435
|
|
|
436
|
## v0.12.0 at 2022-06-19 |
|
437
|
|
|
438
|
Not a release but preparation for changing feature set and API. |
|
439
|
|
|
440
|
|
|
441
|
|
|
442
|
|
|
443
|
|
|
444
|
## v0.11.13 at (Swashplate) 2022-11-10 |
|
445
|
|
|
446
|
The stable bugfix release in memory of [Boris Yuryev](https: |
|
447
|
|
|
448
|
``` |
|
449
|
30 files changed, 405 insertions(+), 136 deletions(-) |
|
450
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
451
|
``` |
|
452
|
|
|
453
|
Fixes: |
|
454
|
|
|
455
|
- Fixed builds with older libc versions after using `fcntl64()` (backport). |
|
456
|
- Fixed builds with older `stdatomic.h` versions, |
|
457
|
where the `ATOMIC_*_LOCK_FREE` macros mistakenly redefined using functions (backport). |
|
458
|
- Added workaround for `mremap()` defect to avoid assertion failure (backport). |
|
459
|
- Workaround for `encryptfs` bug(s) in the `copy_file_range` implementation (backport). |
|
460
|
- Fixed unexpected `MDBX_BUSY` from `mdbx_env_set_option()`, `mdbx_env_set_syncbytes()` |
|
461
|
and `mdbx_env_set_syncperiod()` (backport). |
|
462
|
- CMake requirements lowered to version 3.0.2 (backport). |
|
463
|
|
|
464
|
Minors: |
|
465
|
|
|
466
|
- Minor clarification output of `--help` for `mdbx_test` (backport). |
|
467
|
- Added admonition of insecure for RISC-V (backport). |
|
468
|
- Stochastic scripts and CMake files synchronized with the `devel` branch. |
|
469
|
- Use `--dont-check-ram-size` for small-tests make-targets (backport). |
|
470
|
|
|
471
|
|
|
472
|
## v0.11.12 (Эребуни) at 2022-10-12 |
|
473
|
|
|
474
|
The stable bugfix release. |
|
475
|
|
|
476
|
``` |
|
477
|
11 files changed, 96 insertions(+), 49 deletions(-) |
|
478
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
479
|
``` |
|
480
|
|
|
481
|
Fixes: |
|
482
|
|
|
483
|
- Fixed static assertion failure on platforms where the `off_t` type is wider |
|
484
|
than corresponding fields of `struct flock` used for file locking (backport). |
|
485
|
Now _libmdbx_ will use `fcntl64(F_GETLK64/F_SETLK64/F_SETLKW64)` if available. |
|
486
|
- Fixed assertion check inside `page_retire_ex()` (backport). |
|
487
|
|
|
488
|
Minors: |
|
489
|
|
|
490
|
- Fixed `-Wint-to-pointer-cast` warnings while casting to `mdbx_tid_t` (backport). |
|
491
|
- Removed needless `LockFileEx()` inside `mdbx_env_copy()` (backport). |
|
492
|
|
|
493
|
|
|
494
|
## v0.11.11 (Тендра-1790) at 2022-09-11 |
|
495
|
|
|
496
|
The stable bugfix release. |
|
497
|
|
|
498
|
``` |
|
499
|
10 files changed, 38 insertions(+), 21 deletions(-) |
|
500
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
501
|
``` |
|
502
|
|
|
503
|
Fixes: |
|
504
|
|
|
505
|
- Fixed an extra check for `MDBX_APPENDDUP` inside `mdbx_cursor_put()` which could result in returning `MDBX_EKEYMISMATCH` for valid cases. |
|
506
|
- Fixed an extra ensure/assertion check of `oldest_reader` inside `mdbx_txn_end()`. |
|
507
|
- Fixed derived C++ builds by removing `MDBX_INTERNAL_FUNC` for `mdbx_w2mb()` and `mdbx_mb2w()`. |
|
508
|
|
|
509
|
|
|
510
|
# |
|
511
|
|
|
512
|
The stable bugfix release. |
|
513
|
|
|
514
|
``` |
|
515
|
14 files changed, 263 insertions(+), 252 deletions(-) |
|
516
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
517
|
``` |
|
518
|
|
|
519
|
New: |
|
520
|
|
|
521
|
- The C++ API has been refined to simplify support for `wchar_t` in path names. |
|
522
|
- Added explicit error message for Buildroot's Microblaze toolchain maintainers. |
|
523
|
|
|
524
|
Fixes: |
|
525
|
|
|
526
|
- Never use modern `__cxa_thread_atexit()` on Apple's OSes. |
|
527
|
- Use `MultiByteToWideChar(CP_THREAD_ACP)` instead of `mbstowcs()`. |
|
528
|
- Don't check owner for finished transactions. |
|
529
|
- Fixed typo in `MDBX_EINVAL` which breaks MingGW builds with CLANG. |
|
530
|
|
|
531
|
Minors: |
|
532
|
|
|
533
|
- Fixed variable name typo. |
|
534
|
- Using `ldd` to check used dso. |
|
535
|
- Added `MDBX_WEAK_IMPORT_ATTRIBUTE` macro. |
|
536
|
- Use current transaction geometry for untouched parameters when `env_set_geometry()` called within a write transaction. |
|
537
|
- Minor clarified `iov_page()` failure case. |
|
538
|
|
|
539
|
|
|
540
|
|
|
541
|
## v0.11.9 (Чирчик-1992) at 2022-08-02 |
|
542
|
|
|
543
|
The stable bugfix release. |
|
544
|
|
|
545
|
``` |
|
546
|
18 files changed, 318 insertions(+), 178 deletions(-) |
|
547
|
Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru> |
|
548
|
``` |
|
549
|
|
|
550
|
Acknowledgements: |
|
551
|
|
|
552
|
- [Alex Sharov](https://github.com/AskAlexSharov) and Erigon team for reporting and testing. |
|
553
|
- [Andrew Ashikhmin](https: |
|
554
|
|
|
555
|
New: |
|
556
|
|
|
557
|
- Ability to customise `MDBX_LOCK_SUFFIX`, `MDBX_DATANAME`, `MDBX_LOCKNAME` just by predefine ones during build. |
|
558
|
- Added to [`mdbx::env_managed`](https: |
|
559
|
|
|
560
|
Fixes: |
|
561
|
|
|
562
|
- Fixed hang copy-with-compactification of a corrupted DB |
|
563
|
or in case the volume of output pages is a multiple of `MDBX_ENVCOPY_WRITEBUF`. |
|
564
|
- Fixed standalone non-CMake build on MacOS (`#include AvailabilityMacros.h>`). |
|
565
|
- Fixed unexpected `MDBX_PAGE_FULL` error in rare cases with large database page sizes. |
|
566
|
|
|
567
|
Minors: |
|
568
|
|
|
569
|
- Minor fixes Doxygen references, comments, descriptions, etc. |
|
570
|
- Fixed copy&paste typo inside `meta_checktxnid()`. |
|
571
|
- Minor fix `meta_checktxnid()` to avoid assertion in debug mode. |
|
572
|
- Minor fix `mdbx_env_set_geometry()` to avoid returning `EINVAL` in particular rare cases. |
|
573
|
- Minor refine/fix batch-get testcase for large page size. |
|
574
|
- Added `--pagesize NN` option to long-stotastic test script. |
|
575
|
- Updated Valgrind-suppressions file for modern GCC. |
|
576
|
- Fixed `has no symbols` warning from Apple's ranlib. |
|
577
|
|
|
578
|
|
|
579
|
|
|
580
|
|
|
581
|
|
|
582
|
## v0.11.8 (Baked Apple) at 2022-06-12 |
|
583
|
|
|
584
|
The stable release with an important fixes and workaround for the critical macOS thread-local-storage issue. |
|
585
|
|
|
586
|
Acknowledgements: |
|
587
|
|
|
588
|
- [Masatoshi Fukunaga](https://github.com/mah0x211) for [Lua bindings](https://github.com/mah0x211/lua-libmdbx). |
|
589
|
|
|
590
|
New: |
|
591
|
|
|
592
|
- Added most of transactions flags to the public API. |
|
593
|
- Added `MDBX_NOSUCCESS_EMPTY_COMMIT` build option to return non-success result (`MDBX_RESULT_TRUE`) on empty commit. |
|
594
|
- Reworked validation and import of DBI-handles into a transaction. |
|
595
|
Assumes these changes will be invisible to most users, but will cause fewer surprises in complex DBI cases. |
|
596
|
- Added ability to open DB in without-LCK (exclusive read-only) mode in case no permissions to create/write LCK-file. |
|
597
|
|
|
598
|
Fixes: |
|
599
|
|
|
600
|
- A series of fixes and improvements for automatically generated documentation (Doxygen). |
|
601
|
- Fixed copy&paste bug with could lead to `SIGSEGV` (nullptr dereference) in the exclusive/no-lck mode. |
|
602
|
- Fixed minor warnings from modern Apple's CLANG 13. |
|
603
|
- Fixed minor warnings from CLANG 14 and in-development CLANG 15. |
|
604
|
- Fixed `SIGSEGV` regression in without-LCK (exclusive read-only) mode. |
|
605
|
- Fixed `mdbx_check_fs_local()` for CDROM case on Windows. |
|
606
|
- Fixed nasty typo of typename which caused false `MDBX_CORRUPTED` error in a rare execution path, |
|
607
|
when the size of the thread-ID type not equal to 8. |
|
608
|
- Fixed Elbrus/E2K LCC 1.26 compiler warnings (memory model for atomic operations, etc). |
|
609
|
- Fixed write-after-free memory corruption on latest `macOS` during finalization/cleanup of thread(s) that executed read transaction(s). |
|
610
|
> The issue was suddenly discovered by a [CI](https: |
|
611
|
> after adding an iteration with macOS 11 "Big Sur", and then reproduced on recent release of macOS 12 "Monterey". |
|
612
|
> The issue was never noticed nor reported on macOS 10 "Catalina" nor others. |
|
613
|
> Analysis shown that the problem caused by a change in the behavior of the system library (internals of dyld and pthread) |
|
614
|
> during thread finalization/cleanup: now a memory allocated for a `__thread` variable(s) is released |
|
615
|
> before execution of the registered Thread-Local-Storage destructor(s), |
|
616
|
> thus a TLS-destructor will write-after-free just by legitime dereference any `__thread` variable. |
|
617
|
> This is unexpected crazy-like behavior since the order of resources releasing/destroying |
|
618
|
> is not the reverse of ones acquiring/construction order. Nonetheless such surprise |
|
619
|
> is now workarounded by using atomic compare-and-swap operations on a 64-bit signatures/cookies. |
|
620
|
|
|
621
|
Minors: |
|
622
|
|
|
623
|
- Refined `release-assets` GNU Make target. |
|
624
|
- Added logging to `mdbx_fetch_sdb()` to help debugging complex DBI-handels use cases. |
|
625
|
- Added explicit error message from probe of no-support for `std::filesystem`. |
|
626
|
- Added contributors "score" table by `git fame` to generated docs. |
|
627
|
- Added `mdbx_assert_fail()` to public API (mostly for backtracing). |
|
628
|
- Now C++20 concepts used/enabled only when `__cpp_lib_concepts >= 202002`. |
|
629
|
- Don't provide nor report package information if used as a CMake subproject. |
|
630
|
|
|
631
|
|
|
632
|
|
|
633
|
|
|
634
|
|
|
635
|
## v0.11.7 (Resurrected Sarmat) at 2022-04-22 |
|
636
|
|
|
637
|
The stable risen release after the Github's intentional malicious disaster. |
|
638
|
|
|
639
|
# |
|
640
|
The origin for now is at [GitFlic](https: |
|
641
|
since on 2022-04-15 the Github administration, without any warning nor |
|
642
|
explanation, deleted _libmdbx_ along with a lot of other projects, |
|
643
|
simultaneously blocking access for many developers. |
|
644
|
For the same reason ~~Github~~ is blacklisted forever. |
|
645
|
|
|
646
|
GitFlic already support Russian and English languages, plan to support more, |
|
647
|
including 和 中文. You are welcome! |
|
648
|
|
|
649
|
New: |
|
650
|
|
|
651
|
- Added the `tools-static` make target to build statically linked MDBX tools. |
|
652
|
- Support for Microsoft Visual Studio 2022. |
|
653
|
- Support build by MinGW' make from command line without CMake. |
|
654
|
- Added `mdbx::filesystem` C++ API namespace that corresponds to `std::filesystem` or `std::experimental::filesystem`. |
|
655
|
- Created [website](https: |
|
656
|
- Used `https://web.archive.org/web/https://github.com/erthink/libmdbx` for dead (or temporarily lost) resources deleted by ~~Github~~. |
|
657
|
- Added `--loglevel=` command-line option to the `mdbx_test` tool. |
|
658
|
- Added few fast smoke-like tests into CMake builds. |
|
659
|
|
|
660
|
Fixes: |
|
661
|
|
|
662
|
- Fixed a race between starting a transaction and creating a DBI descriptor that could lead to `SIGSEGV` in the cursor tracking code. |
|
663
|
- Clarified description of `MDBX_EPERM` error returned from `mdbx_env_set_geometry()`. |
|
664
|
- Fixed non-promoting the parent transaction to be dirty in case the undo of the geometry update failed during abortion of a nested transaction. |
|
665
|
- Resolved linking issues with `libstdc++fs`/`libc++fs`/`libc++experimental` for C++ `std::filesystem` or `std::experimental::filesystem` for legacy compilers. |
|
666
|
- Added workaround for GNU Make 3.81 and earlier. |
|
667
|
- Added workaround for Elbrus/LCC 1.25 compiler bug of class inline `static constexpr` member field. |
|
668
|
- [Fixed](https: |
|
669
|
- Fixed detection of `C++20` concepts accessibility. |
|
670
|
- Fixed detection of Clang's LTO availability for Android. |
|
671
|
- Fixed extra definition of `_FILE_OFFSET_BITS=64` for Android that is problematic for 32-bit Bionic. |
|
672
|
- Fixed build for ARM/ARM64 by MSVC. |
|
673
|
- Fixed non-x86 Windows builds with `MDBX_WITHOUT_MSVC_CRT=ON` and `MDBX_BUILD_SHARED_LIBRARY=ON`. |
|
674
|
|
|
675
|
Minors: |
|
676
|
|
|
677
|
- Resolve minor MSVC warnings: avoid `/INCREMENTAL[:YES]` with `/LTCG`, `/W4` with `/W3`, the `C5105` warning. |
|
678
|
- Switched to using `MDBX_EPERM` instead of `MDBX_RESULT_TRUE` to indicate that the geometry cannot be updated. |
|
679
|
- Added `NULL` checking during memory allocation inside `mdbx_chk`. |
|
680
|
- Resolved all warnings from MinGW while used without CMake. |
|
681
|
- Added inheretable `target_include_directories()` to `CMakeLists.txt` for easy integration. |
|
682
|
- Added build-time checks and paranoid runtime assertions for the `off_t` arguments of `fcntl()` which are used for locking. |
|
683
|
- Added `-Wno-lto-type-mismatch` to avoid false-positive warnings from old GCC during LTO-enabled builds. |
|
684
|
- Added checking for TID (system thread id) to avoid hang on 32-bit Bionic/Android within `pthread_mutex_lock()`. |
|
685
|
- Reworked `MDBX_BUILD_TARGET` of CMake builds. |
|
686
|
- Added `CMAKE_HOST_ARCH` and `CMAKE_HOST_CAN_RUN_EXECUTABLES_BUILT_FOR_TARGET`. |
|
687
|
|
|
688
|
|
|
689
|
|
|
690
|
|
|
691
|
|
|
692
|
## v0.11.6 at 2022-03-24 |
|
693
|
|
|
694
|
The stable release with the complete workaround for an incoherence flaw of Linux unified page/buffer cache. |
|
695
|
Nonetheless the cause for this trouble may be an issue of Intel CPU cache/MESI. |
|
696
|
See [issue |
|
697
|
|
|
698
|
Acknowledgements: |
|
699
|
|
|
700
|
- [David Bouyssié](https://github.com/david-bouyssie) for [Scala bindings](https://github.com/david-bouyssie/mdbx4s). |
|
701
|
- [Michelangelo Riccobene](https://github.com/mriccobene) for reporting and testing. |
|
702
|
|
|
703
|
Fixes: |
|
704
|
|
|
705
|
- [Added complete workaround](https: |
|
706
|
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/272) cursor reusing for read-only transactions. |
|
707
|
- Fixed copy&paste typo inside `mdbx::cursor::find_multivalue()`. |
|
708
|
|
|
709
|
Minors: |
|
710
|
|
|
711
|
- Minor refine C++ API for convenience. |
|
712
|
- Minor internals refines. |
|
713
|
- Added `lib-static` and `lib-shared` targets for make. |
|
714
|
- Added minor workaround for AppleClang 13.3 bug. |
|
715
|
- Clarified error messages of a signature/version mismatch. |
|
716
|
|
|
717
|
|
|
718
|
## v0.11.5 at 2022-02-23 |
|
719
|
|
|
720
|
The release with the temporary hotfix for a flaw of Linux unified page/buffer cache. |
|
721
|
See [issue |
|
722
|
|
|
723
|
Acknowledgements: |
|
724
|
|
|
725
|
- [Simon Leier](https://github.com/leisim) for reporting and testing. |
|
726
|
- [Kai Wetlesen](https://github.com/kaiwetlesen) for [RPMs](http://copr.fedorainfracloud.org/coprs/kwetlesen/libmdbx/). |
|
727
|
- [Tullio Canepa](https://github.com/canepat) for reporting C++ API issue and contributing. |
|
728
|
|
|
729
|
Fixes: |
|
730
|
|
|
731
|
- [Added hotfix](https: |
|
732
|
- [Fixed/Reworked](https://libmdbx.dqdkfa.ru/dead-github/pull/270) move-assignment operators for "managed" classes of C++ API. |
|
733
|
- Fixed potential `SIGSEGV` while open DB with overrided non-default page size. |
|
734
|
- [Made](https://libmdbx.dqdkfa.ru/dead-github/issues/267) `mdbx_env_open()` idempotence in failure cases. |
|
735
|
- Refined/Fixed pages reservation inside `mdbx_update_gc()` to avoid non-reclamation in a rare cases. |
|
736
|
- Fixed typo in a retained space calculation for the hsr-callback. |
|
737
|
|
|
738
|
Minors: |
|
739
|
|
|
740
|
- Reworked functions for meta-pages, split-off non-volatile. |
|
741
|
- Disentangled C11-atomic fences/barriers and pure-functions (with `__attribute__((__pure__))`) to avoid compiler misoptimization. |
|
742
|
- Fixed hypotetic unaligned access to 64-bit dwords on ARM with `__ARM_FEATURE_UNALIGNED` defined. |
|
743
|
- Reasonable paranoia that makes clarity for code readers. |
|
744
|
- Minor fixes Doxygen references, comments, descriptions, etc. |
|
745
|
|
|
746
|
|
|
747
|
## v0.11.4 at 2022-02-02 |
|
748
|
|
|
749
|
The stable release with fixes for large and huge databases sized of 4..128 TiB. |
|
750
|
|
|
751
|
Acknowledgements: |
|
752
|
|
|
753
|
- [Ledgerwatch](https://github.com/ledgerwatch), [Binance](https://github.com/binance-chain) and [Positive Technologies](https://www.ptsecurity.com/) teams for reporting, assistance in investigation and testing. |
|
754
|
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting, testing and provide resources for remote debugging/investigation. |
|
755
|
- [Kris Zyp](https://github.com/kriszyp) for [Deno](https://deno.land/) support. |
|
756
|
|
|
757
|
New features, extensions and improvements: |
|
758
|
|
|
759
|
- Added treating the `UINT64_MAX` value as maximum for given option inside `mdbx_env_set_option()`. |
|
760
|
- Added `to_hex/to_base58/to_base64::output(std::ostream&)` overloads without using temporary string objects as buffers. |
|
761
|
- Added `--geometry-jitter=YES|no` option to the test framework. |
|
762
|
- Added support for [Deno](https: |
|
763
|
|
|
764
|
Fixes: |
|
765
|
|
|
766
|
- Fixed handling `MDBX_opt_rp_augment_limit` for GC's records from huge transactions (Erigon/Akula/Ethereum). |
|
767
|
- [Fixed](https: |
|
768
|
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/pull/261) missing copy assignment operator for `mdbx::move_result`. |
|
769
|
- Fixed missing `&` for `std::ostream &operator<<()` overloads. |
|
770
|
- Fixed unexpected `EXDEV` (Cross-device link) error from `mdbx_env_copy()`. |
|
771
|
- Fixed base64 encoding/decoding bugs in auxillary C++ API. |
|
772
|
- Fixed overflow of `pgno_t` during checking PNL on 64-bit platforms. |
|
773
|
- [Fixed](https: |
|
774
|
- Reworked checking `MAX_PAGENO` and DB upper-size geometry limit. |
|
775
|
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/265) build for some combinations of versions of MSVC and Windows SDK. |
|
776
|
|
|
777
|
Minors: |
|
778
|
|
|
779
|
- Added workaround for CLANG bug [D79919/PR42445](https://reviews.llvm.org/D79919). |
|
780
|
- Fixed build test on Android (using `pthread_barrier_t` stub). |
|
781
|
- Disabled C++20 concepts for CLANG < 14 on Android. |
|
782
|
- Fixed minor `unused parameter` warning. |
|
783
|
- Added CI for Android. |
|
784
|
- Refine/cleanup internal logging. |
|
785
|
- Refined line splitting inside hex/base58/base64 encoding to avoid `\n` at the end. |
|
786
|
- Added workaround for modern libstdc++ with CLANG < 4.x |
|
787
|
- Relaxed txn-check rules for auxiliary functions. |
|
788
|
- Clarified a comments and descriptions, etc. |
|
789
|
- Using the `-fno-semantic interposition` option to reduce the overhead to calling self own public functions. |
|
790
|
|
|
791
|
|
|
792
|
## v0.11.3 at 2021-12-31 |
|
793
|
|
|
794
|
Acknowledgements: |
|
795
|
|
|
796
|
- [gcxfd <i@rmw.link>](https://github.com/gcxfd) for reporting, contributing and testing. |
|
797
|
- [장세연 (Чан Се Ен)](https://github.com/sasgas) for reporting and testing. |
|
798
|
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting, testing and provide resources for remote debugging/investigation. |
|
799
|
|
|
800
|
New features, extensions and improvements: |
|
801
|
|
|
802
|
- [Added](https: |
|
803
|
- [Added](https: |
|
804
|
- C++ API is finalized now. |
|
805
|
- The GC update stage has been [significantly speeded](https: |
|
806
|
|
|
807
|
Fixes: |
|
808
|
|
|
809
|
- Disabled C++20 concepts for stupid AppleClang 13.x |
|
810
|
- Fixed internal collision of `MDBX_SHRINK_ALLOWED` with `MDBX_ACCEDE`. |
|
811
|
|
|
812
|
Minors: |
|
813
|
|
|
814
|
- Fixed returning `MDBX_RESULT_TRUE` (unexpected -1) from `mdbx_env_set_option()`. |
|
815
|
- Added `mdbx_env_get_syncbytes()` and `mdbx_env_get_syncperiod()`. |
|
816
|
- [Clarified](https: |
|
817
|
- Reworked/simplified `mdbx_env_sync_internal()`. |
|
818
|
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/248) extra assertion inside `mdbx_cursor_put()` for `MDBX_DUPFIXED` cases. |
|
819
|
- Avoiding extra looping inside `mdbx_env_info_ex()`. |
|
820
|
- Explicitly enabled core dumps from stochastic tests scripts on Linux. |
|
821
|
- [Fixed](https://libmdbx.dqdkfa.ru/dead-github/issues/253) `mdbx_override_meta()` to avoid false-positive assertions. |
|
822
|
- For compatibility reverted returning `MDBX_ENODATA`for some cases. |
|
823
|
|
|
824
|
|
|
825
|
## v0.11.2 at 2021-12-02 |
|
826
|
|
|
827
|
Acknowledgements: |
|
828
|
|
|
829
|
- [장세연 (Чан Се Ен)](https: |
|
830
|
- [Alain Picard](https://github.com/castortech) for [Java bindings](https://github.com/castortech/mdbxjni). |
|
831
|
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing. |
|
832
|
- [Kris Zyp](https://github.com/kriszyp) for reporting and testing. |
|
833
|
- [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/libmdbx-rs). |
|
834
|
|
|
835
|
Fixes: |
|
836
|
|
|
837
|
- [Fixed compilation](https: |
|
838
|
- [Fixed unexpected `MDBX_PROBLEM` error](https: |
|
839
|
- [Fixed returning `MDBX_NOTFOUND` error](https://libmdbx.dqdkfa.ru/dead-github/issues/243) in case an inexact value found for `MDBX_GET_BOTH` operation. |
|
840
|
- [Fixed compilation](https: |
|
841
|
|
|
842
|
Minors: |
|
843
|
|
|
844
|
- Fixed `constexpr`-related macros for legacy compilers. |
|
845
|
- Allowed to define 'CMAKE_CXX_STANDARD` using an environment variable. |
|
846
|
- Simplified collection statistics of page operation . |
|
847
|
- Added `MDBX_FORCE_BUILD_AS_MAIN_PROJECT` cmake option. |
|
848
|
- Remove unneeded `#undef P_DIRTY`. |
|
849
|
|
|
850
|
|
|
851
|
## v0.11.1 at 2021-10-23 |
|
852
|
|
|
853
|
# |
|
854
|
|
|
855
|
The database format signature has been changed to prevent |
|
856
|
forward-interoperability with an previous releases, which may lead to a |
|
857
|
[false positive diagnosis of database corruption](https: |
|
858
|
due to flaws of an old library versions. |
|
859
|
|
|
860
|
This change is mostly invisible: |
|
861
|
|
|
862
|
- previously versions are unable to read/write a new DBs; |
|
863
|
- but the new release is able to handle an old DBs and will silently upgrade ones. |
|
864
|
|
|
865
|
Acknowledgements: |
|
866
|
|
|
867
|
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing. |
|
868
|
|
|
869
|
|
|
870
|
|
|
871
|
|
|
872
|
|
|
873
|
## v0.10.5 at 2021-10-13 (obsolete, please use v0.11.1) |
|
874
|
|
|
875
|
Unfortunately, the `v0.10.5` accidentally comes not full-compatible with previous releases: |
|
876
|
|
|
877
|
- `v0.10.5` can read/processing DBs created by previous releases, i.e. the backward-compatibility is provided; |
|
878
|
- however, previous releases may lead to false-corrupted state with DB that was touched by `v0.10.5`, i.e. the forward-compatibility is broken for `v0.10.4` and earlier. |
|
879
|
|
|
880
|
This cannot be fixed, as it requires fixing past versions, which as a result we will just get a current version. |
|
881
|
Therefore, it is recommended to use `v0.11.1` instead of `v0.10.5`. |
|
882
|
|
|
883
|
Acknowledgements: |
|
884
|
|
|
885
|
- [Noel Kuntze](https: |
|
886
|
|
|
887
|
Fixes: |
|
888
|
|
|
889
|
- Fixed unaligned access regression after the `#pragma pack` fix for modern compilers. |
|
890
|
- Added UBSAN-test to CI to avoid a regression(s) similar to lately fixed. |
|
891
|
- Fixed possibility of meta-pages clashing after manually turn to a particular meta-page using `mdbx_chk` utility. |
|
892
|
|
|
893
|
Minors: |
|
894
|
|
|
895
|
- Refined handling of weak or invalid meta-pages while a DB opening. |
|
896
|
- Refined providing information for the `@MAIN` and `@GC` sub-databases of a last committed modification transaction's ID. |
|
897
|
|
|
898
|
|
|
899
|
## v0.10.4 at 2021-10-10 |
|
900
|
|
|
901
|
Acknowledgements: |
|
902
|
|
|
903
|
- [Artem Vorotnikov](https://github.com/vorot93) for support [Rust wrapper](https://github.com/vorot93/libmdbx-rs). |
|
904
|
- [Andrew Ashikhmin](https: |
|
905
|
|
|
906
|
Fixes: |
|
907
|
|
|
908
|
- Fixed possibility of looping update GC during transaction commit (no public issue since the problem was discovered inside [Positive Technologies](https: |
|
909
|
- Fixed `#pragma pack` to avoid provoking some compilers to generate code with [unaligned access](https: |
|
910
|
- Fixed `noexcept` for potentially throwing `txn::put()` of C++ API. |
|
911
|
|
|
912
|
Minors: |
|
913
|
|
|
914
|
- Added stochastic test script for checking small transactions cases. |
|
915
|
- Removed extra transaction commit/restart inside test framework. |
|
916
|
- In debugging builds fixed a too small (single page) by default DB shrink threshold. |
|
917
|
|
|
918
|
|
|
919
|
## v0.10.3 at 2021-08-27 |
|
920
|
|
|
921
|
Acknowledgements: |
|
922
|
|
|
923
|
- [Francisco Vallarino](https://github.com/fjvallarino) for [Haskell bindings for libmdbx](https://hackage.haskell.org/package/libmdbx). |
|
924
|
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing. |
|
925
|
- [Andrea Lanfranchi](https: |
|
926
|
|
|
927
|
Extensions and improvements: |
|
928
|
|
|
929
|
- Added `cursor::erase()` overloads for `key` and for `key-value`. |
|
930
|
- Resolve minor Coverity Scan issues (no fixes but some hint/comment were added). |
|
931
|
- Resolve minor UndefinedBehaviorSanitizer issues (no fixes but some workaround were added). |
|
932
|
|
|
933
|
Fixes: |
|
934
|
|
|
935
|
- Always setup `madvise` while opening DB (fixes https: |
|
936
|
- Fixed checking legacy `P_DIRTY` flag (`0x10`) for nested/sub-pages. |
|
937
|
|
|
938
|
Minors: |
|
939
|
|
|
940
|
- Fixed getting revision number from middle of history during amalgamation (GNU Makefile). |
|
941
|
- Fixed search GCC tools for LTO (CMake scripts). |
|
942
|
- Fixed/reorder dirs list for search CLANG tools for LTO (CMake scripts). |
|
943
|
- Fixed/workarounds for CLANG < 9.x |
|
944
|
- Fixed CMake warning about compatibility with 3.8.2 |
|
945
|
|
|
946
|
|
|
947
|
## v0.10.2 at 2021-07-26 |
|
948
|
|
|
949
|
Acknowledgements: |
|
950
|
|
|
951
|
- [Alex Sharov](https://github.com/AskAlexSharov) for reporting and testing. |
|
952
|
- [Andrea Lanfranchi](https: |
|
953
|
- [Lionel Debroux](https://github.com/debrouxl) for fuzzing tests and reporting bugs. |
|
954
|
- [Sergey Fedotov](https://github.com/SergeyFromHell/) for [`node-mdbx` NodeJS bindings](https://www.npmjs.com/package/node-mdbx). |
|
955
|
- [Kris Zyp](https://github.com/kriszyp) for [`lmdbx-store` NodeJS bindings](https://github.com/kriszyp/lmdbx-store). |
|
956
|
- [Noel Kuntze](https://github.com/Thermi) for [draft Python bindings](https://libmdbx.dqdkfa.ru/dead-github/commits/python-bindings). |
|
957
|
|
|
958
|
New features, extensions and improvements: |
|
959
|
|
|
960
|
- [Allow to predefine/override `MDBX_BUILD_TIMESTAMP` for builds reproducibility](https: |
|
961
|
- Added options support for `long-stochastic` script. |
|
962
|
- Avoided `MDBX_TXN_FULL` error for large transactions when possible. |
|
963
|
- The `MDBX_READERS_LIMIT` increased to `32767`. |
|
964
|
- Raise `MDBX_TOO_LARGE` under Valgrind/ASAN if being opened DB is 100 larger than RAM (to avoid hangs and OOM). |
|
965
|
- Minimized the size of poisoned/unpoisoned regions to avoid Valgrind/ASAN stuck. |
|
966
|
- Added more workarounds for QEMU for testing builds for 32-bit platforms, Alpha and Sparc architectures. |
|
967
|
- `mdbx_chk` now skips iteration & checking of DB' records if corresponding page-tree is corrupted (to avoid `SIGSEGV`, ASAN failures, etc). |
|
968
|
- Added more checks for [rare/fuzzing corruption cases](https://libmdbx.dqdkfa.ru/dead-github/issues/217). |
|
969
|
|
|
970
|
Backward compatibility break: |
|
971
|
|
|
972
|
- Use file `VERSION.txt` for version information instead of `VERSION` to avoid collision with `#include <version>`. |
|
973
|
- Rename `slice::from/to_FOO_bytes()` to `slice::envisage_from/to_FOO_length()'. |
|
974
|
- Rename `MDBX_TEST_EXTRA` make's variable to `MDBX_SMOKE_EXTRA`. |
|
975
|
- Some details of the C++ API have been changed for subsequent freezing. |
|
976
|
|
|
977
|
Fixes: |
|
978
|
|
|
979
|
- Fixed excess meta-pages checks in case `mdbx_chk` is called to check the DB for a specific meta page and thus could prevent switching to the selected meta page, even if the check passed without errors. |
|
980
|
- Fixed [recursive use of SRW-lock on Windows cause by `MDBX_NOTLS` option](https: |
|
981
|
- Fixed [log a warning during a new DB creation](https: |
|
982
|
- Fixed [false-negative `mdbx_cursor_eof()` result](https: |
|
983
|
- Fixed [`make install` with non-GNU `install` utility (OSX, BSD)](https: |
|
984
|
- Fixed [installation by `CMake` in special cases by complete use `GNUInstallDirs`'s variables](https://libmdbx.dqdkfa.ru/dead-github/issues/209). |
|
985
|
- Fixed [C++ Buffer issue with `std::string` and alignment](https: |
|
986
|
- Fixed `safe64_reset()` for platforms without atomic 64-bit compare-and-swap. |
|
987
|
- Fixed hang/shutdown on big-endian platforms without `__cxa_thread_atexit()`. |
|
988
|
- Fixed [using bad meta-pages if DB was partially/recoverable corrupted](https://libmdbx.dqdkfa.ru/dead-github/issues/217). |
|
989
|
- Fixed extra `noexcept` for `buffer::&assign_reference()`. |
|
990
|
- Fixed `bootid` generation on Windows for case of change system' time. |
|
991
|
- Fixed [test framework keygen-related issue](https: |
|
992
|
|
|
993
|
|
|
994
|
## v0.10.1 at 2021-06-01 |
|
995
|
|
|
996
|
Acknowledgements: |
|
997
|
|
|
998
|
- [Alexey Akhunov](https://github.com/AlexeyAkhunov) and [Alex Sharov](https://github.com/AskAlexSharov) for bug reporting and testing. |
|
999
|
- [Andrea Lanfranchi](https://github.com/AndreaLanfranchi) for bug reporting and testing related to WSL2. |
|
1,000
|
|
|
1,001
|
New features: |
|
1,002
|
|
|
1,003
|
- Added `-p` option to `mdbx_stat` utility for printing page operations statistic. |
|
1,004
|
- Added explicit checking for and warning about using unfit github's archives. |
|
1,005
|
- Added fallback from [OFD locking](https: |
|
1,006
|
- Added [Plan 9](https: |
|
1,007
|
- Support for opening from WSL2 environment a DB hosted on Windows drive and mounted via [DrvFs](https: |
|
1,008
|
|
|
1,009
|
Fixes: |
|
1,010
|
|
|
1,011
|
- Fixed minor "foo not used" warnings from modern C++ compilers when building the C++ part of the library. |
|
1,012
|
- Fixed confusing/messy errors when build library from unfit github's archives (https://libmdbx.dqdkfa.ru/dead-github/issues/197). |
|
1,013
|
- Fixed ` |
|
1,014
|
- Fixed rare unexpected `MDBX_PROBLEM` error during altering data in huge transactions due to wrong spilling/oust of dirty pages (https://libmdbx.dqdkfa.ru/dead-github/issues/195). |
|
1,015
|
- Re-Fixed WSL1/WSL2 detection with distinguishing (https://libmdbx.dqdkfa.ru/dead-github/issues/97). |
|
1,016
|
|
|
1,017
|
|
|
1,018
|
## v0.10.0 at 2021-05-09 |
|
1,019
|
|
|
1,020
|
Acknowledgements: |
|
1,021
|
|
|
1,022
|
- [Mahlon E. Smith](https://github.com/mahlonsmith) for [Ruby bindings](https://rubygems.org/gems/mdbx/). |
|
1,023
|
- [Alex Sharov](https://github.com/AskAlexSharov) for [mdbx-go](https://github.com/torquem-ch/mdbx-go), bug reporting and testing. |
|
1,024
|
- [Artem Vorotnikov](https://github.com/vorot93) for bug reporting and PR. |
|
1,025
|
- [Paolo Rebuffo](https://www.linkedin.com/in/paolo-rebuffo-8255766/), [Alexey Akhunov](https://github.com/AlexeyAkhunov) and Mark Grosberg for donations. |
|
1,026
|
- [Noel Kuntze](https://github.com/Thermi) for preliminary [Python bindings](https://github.com/Thermi/libmdbx/tree/python-bindings) |
|
1,027
|
|
|
1,028
|
New features: |
|
1,029
|
|
|
1,030
|
- Added `mdbx_env_set_option()` and `mdbx_env_get_option()` for controls |
|
1,031
|
various runtime options for an environment (announce of this feature was missed in a previous news). |
|
1,032
|
- Added `MDBX_DISABLE_PAGECHECKS` build option to disable some checks to reduce an overhead |
|
1,033
|
and detection probability of database corruption to a values closer to the LMDB. |
|
1,034
|
The `MDBX_DISABLE_PAGECHECKS=1` provides a performance boost of about 10% in CRUD scenarios, |
|
1,035
|
and conjointly with the `MDBX_ENV_CHECKPID=0` and `MDBX_TXN_CHECKOWNER=0` options can yield |
|
1,036
|
up to 30% more performance compared to LMDB. |
|
1,037
|
- Using float point (exponential quantized) representation for internal 16-bit values |
|
1,038
|
of grow step and shrink threshold when huge ones (https://libmdbx.dqdkfa.ru/dead-github/issues/166). |
|
1,039
|
To minimize the impact on compatibility, only the odd values inside the upper half |
|
1,040
|
of the range (i.e. 32769..65533) are used for the new representation. |
|
1,041
|
- Added the `mdbx_drop` similar to LMDB command-line tool to purge or delete (sub)database(s). |
|
1,042
|
- [Ruby bindings](https://rubygems.org/gems/mdbx/) is available now by [Mahlon E. Smith](https://github.com/mahlonsmith). |
|
1,043
|
- Added `MDBX_ENABLE_MADVISE` build option which controls the use of POSIX `madvise()` hints and friends. |
|
1,044
|
- The internal node sizes were refined, resulting in a reduction in large/overflow pages in some use cases |
|
1,045
|
and a slight increase in limits for a keys size to ≈½ of page size. |
|
1,046
|
- Added to `mdbx_chk` output number of keys/items on pages. |
|
1,047
|
- Added explicit `install-strip` and `install-no-strip` targets to the `Makefile` (https://libmdbx.dqdkfa.ru/dead-github/pull/180). |
|
1,048
|
- Major rework page splitting (af9b7b560505684249b76730997f9e00614b8113) for |
|
1,049
|
- An "auto-appending" feature upon insertion for both ascending and |
|
1,050
|
descending key sequences. As a result, the optimality of page filling |
|
1,051
|
increases significantly (more densely, less slackness) while |
|
1,052
|
inserting ordered sequences of keys, |
|
1,053
|
- A "splitting at middle" to make page tree more balanced on average. |
|
1,054
|
- Added `mdbx_get_sysraminfo()` to the API. |
|
1,055
|
- Added guessing a reasonable maximum DB size for the default upper limit of geometry (https: |
|
1,056
|
- Major rework internal labeling of a dirty pages (958fd5b9479f52f2124ab7e83c6b18b04b0e7dda) for |
|
1,057
|
a "transparent spilling" feature with the gist to make a dirty pages |
|
1,058
|
be ready to spilling (writing to a disk) without further altering ones. |
|
1,059
|
Thus in the `MDBX_WRITEMAP` mode the OS kernel able to oust dirty pages |
|
1,060
|
to DB file without further penalty during transaction commit. |
|
1,061
|
As a result, page swapping and I/O could be significantly reduced during extra large transactions and/or lack of memory. |
|
1,062
|
- Minimized reading leaf-pages during dropping subDB(s) and nested trees. |
|
1,063
|
- Major rework a spilling of dirty pages to support [LRU](https: |
|
1,064
|
policy and prioritization for a large/overflow pages. |
|
1,065
|
- Statistics of page operations (split, merge, copy, spill, etc) now available through `mdbx_env_info_ex()`. |
|
1,066
|
- Auto-setup limit for length of dirty pages list (`MDBX_opt_txn_dp_limit` option). |
|
1,067
|
- Support `make options` to list available build options. |
|
1,068
|
- Support `make help` to list available make targets. |
|
1,069
|
- Silently `make`'s build by default. |
|
1,070
|
- Preliminary [Python bindings](https: |
|
1,071
|
by [Noel Kuntze](https: |
|
1,072
|
|
|
1,073
|
Backward compatibility break: |
|
1,074
|
|
|
1,075
|
- The `MDBX_AVOID_CRT` build option was renamed to `MDBX_WITHOUT_MSVC_CRT`. |
|
1,076
|
This option is only relevant when building for Windows. |
|
1,077
|
- The `mdbx_env_stat()` always, and `mdbx_env_stat_ex()` when called with the zeroed transaction parameter, |
|
1,078
|
now internally start temporary read transaction and thus may returns `MDBX_BAD_RSLOT` error. |
|
1,079
|
So, just never use deprecated `mdbx_env_stat()' and call `mdbx_env_stat_ex()` with transaction parameter. |
|
1,080
|
- The build option `MDBX_CONFIG_MANUAL_TLS_CALLBACK` was removed and now just a non-zero value of |
|
1,081
|
the `MDBX_MANUAL_MODULE_HANDLER` macro indicates the requirement to manually call `mdbx_module_handler()` |
|
1,082
|
when loading libraries and applications uses statically linked libmdbx on an obsolete Windows versions. |
|
1,083
|
|
|
1,084
|
Fixes: |
|
1,085
|
|
|
1,086
|
- Fixed performance regression due non-optimal C11 atomics usage (https: |
|
1,087
|
- Fixed "reincarnation" of subDB after it deletion (https: |
|
1,088
|
- Fixed (disallowing) implicit subDB deletion via operations on `@MAIN`'s DBI-handle. |
|
1,089
|
- Fixed a crash of `mdbx_env_info_ex()` in case of a call for a non-open environment (https: |
|
1,090
|
- Fixed the selecting/adjustment values inside `mdbx_env_set_geometry()` for implicit out-of-range cases (https: |
|
1,091
|
- Fixed `mdbx_env_set_option()` for set initial and limit size of dirty page list ((https: |
|
1,092
|
- Fixed an unreasonably huge default upper limit for DB geometry (https: |
|
1,093
|
- Fixed `constexpr` specifier for the `slice::invalid()`. |
|
1,094
|
- Fixed (no)readahead auto-handling (https: |
|
1,095
|
- Fixed non-alloy build for Windows. |
|
1,096
|
- Switched to using Heap-functions instead of LocalAlloc/LocalFree on Windows. |
|
1,097
|
- Fixed `mdbx_env_stat_ex()` to returning statistics of the whole environment instead of MainDB only (https: |
|
1,098
|
- Fixed building by GCC 4.8.5 (added workaround for a preprocessor's bug). |
|
1,099
|
- Fixed building C++ part for iOS <= 13.0 (unavailability of `std::filesystem::path`). |
|
1,100
|
- Fixed building for Windows target versions prior to Windows Vista (`WIN32_WINNT < 0x0600`). |
|
1,101
|
- Fixed building by MinGW for Windows (https: |
|
1,102
|
|
|
1,103
|
|
|
1,104
|
|
|
1,105
|
|
|
1,106
|
|
|
1,107
|
## v0.9.3 at 2021-02-02 |
|
1,108
|
|
|
1,109
|
Acknowledgements: |
|
1,110
|
|
|
1,111
|
- [Mahlon E. Smith](http://www.martini.nu/) for [FreeBSD port of libmdbx](https://svnweb.freebsd.org/ports/head/databases/mdbx/). |
|
1,112
|
- [장세연](http: |
|
1,113
|
- [Clément Renault](https://github.com/Kerollmops/heed) for [Heed](https://github.com/Kerollmops/heed) fully typed Rust wrapper. |
|
1,114
|
- [Alex Sharov](https: |
|
1,115
|
- [Noel Kuntze](https: |
|
1,116
|
|
|
1,117
|
Removed options and features: |
|
1,118
|
|
|
1,119
|
- Drop `MDBX_HUGE_TRANSACTIONS` build-option (now no longer required). |
|
1,120
|
|
|
1,121
|
New features: |
|
1,122
|
|
|
1,123
|
- Package for FreeBSD is available now by Mahlon E. Smith. |
|
1,124
|
- New API functions to get/set various options (https://libmdbx.dqdkfa.ru/dead-github/issues/128): |
|
1,125
|
- the maximum number of named databases for the environment; |
|
1,126
|
- the maximum number of threads/reader slots; |
|
1,127
|
- threshold (since the last unsteady commit) to force flush the data buffers to disk; |
|
1,128
|
- relative period (since the last unsteady commit) to force flush the data buffers to disk; |
|
1,129
|
- limit to grow a list of reclaimed/recycled page's numbers for finding a sequence of contiguous pages for large data items; |
|
1,130
|
- limit to grow a cache of dirty pages for reuse in the current transaction; |
|
1,131
|
- limit of a pre-allocated memory items for dirty pages; |
|
1,132
|
- limit of dirty pages for a write transaction; |
|
1,133
|
- initial allocation size for dirty pages list of a write transaction; |
|
1,134
|
- maximal part of the dirty pages may be spilled when necessary; |
|
1,135
|
- minimal part of the dirty pages should be spilled when necessary; |
|
1,136
|
- how much of the parent transaction dirty pages will be spilled while start each child transaction; |
|
1,137
|
- Unlimited/Dynamic size of retired and dirty page lists (https://libmdbx.dqdkfa.ru/dead-github/issues/123). |
|
1,138
|
- Added `-p` option (purge subDB before loading) to `mdbx_load` tool. |
|
1,139
|
- Reworked spilling of large transaction and committing of nested transactions: |
|
1,140
|
- page spilling code reworked to avoid the flaws and bugs inherited from LMDB; |
|
1,141
|
- limit for number of dirty pages now is controllable at runtime; |
|
1,142
|
- a spilled pages, including overflow/large pages, now can be reused and refunded/compactified in nested transactions; |
|
1,143
|
- more effective refunding/compactification especially for the loosed page cache. |
|
1,144
|
- Added `MDBX_ENABLE_REFUND` and `MDBX_PNL_ASCENDING` internal/advanced build options. |
|
1,145
|
- Added `mdbx_default_pagesize()` function. |
|
1,146
|
- Better support architectures with a weak/relaxed memory consistency model (ARM, AARCH64, PPC, MIPS, RISC-V, etc) by means [C11 atomics](https://en.cppreference.com/w/c/atomic). |
|
1,147
|
- Speed up page number lists and dirty page lists (https: |
|
1,148
|
- Added `LIBMDBX_NO_EXPORTS_LEGACY_API` build option. |
|
1,149
|
|
|
1,150
|
Fixes: |
|
1,151
|
|
|
1,152
|
- Fixed missing cleanup (null assigned) in the C++ commit/abort (https://libmdbx.dqdkfa.ru/dead-github/pull/143). |
|
1,153
|
- Fixed `mdbx_realloc()` for case of nullptr and `MDBX_WITHOUT_MSVC_CRT=ON` for Windows. |
|
1,154
|
- Fixed the possibility to use invalid and renewed (closed & re-opened, dropped & re-created) DBI-handles (https: |
|
1,155
|
- Fixed 4-byte aligned access to 64-bit integers, including access to the `bootid` meta-page's field (https: |
|
1,156
|
- Fixed minor/potential memory leak during page flushing and unspilling. |
|
1,157
|
- Fixed handling states of cursors's and subDBs's for nested transactions. |
|
1,158
|
- Fixed page leak in extra rare case the list of retired pages changed during update GC on transaction commit. |
|
1,159
|
- Fixed assertions to avoid false-positive UB detection by CLANG/LLVM (https://libmdbx.dqdkfa.ru/dead-github/issues/153). |
|
1,160
|
- Fixed `MDBX_TXN_FULL` and regressive `MDBX_KEYEXIST` during large transaction commit with `MDBX_LIFORECLAIM` (https: |
|
1,161
|
- Fixed auto-recovery (`weak->steady` with the same boot-id) when Database size at last weak checkpoint is large than at last steady checkpoint. |
|
1,162
|
- Fixed operation on systems with unusual small/large page size, including PowerPC (https://libmdbx.dqdkfa.ru/dead-github/issues/157). |
|
1,163
|
|
|
1,164
|
|
|
1,165
|
## v0.9.2 at 2020-11-27 |
|
1,166
|
|
|
1,167
|
Acknowledgements: |
|
1,168
|
|
|
1,169
|
- Jens Alfke (Mobile Architect at [Couchbase](https://www.couchbase.com/)) for [NimDBX](https://github.com/snej/nimdbx). |
|
1,170
|
- Clément Renault (CTO at [MeiliSearch](https://www.meilisearch.com/)) for [mdbx-rs](https://github.com/Kerollmops/mdbx-rs). |
|
1,171
|
- Alex Sharov (Go-Lang Teach Lead at [TurboGeth/Ethereum](https://ethereum.org/)) for an extreme test cases and bug reporting. |
|
1,172
|
- George Hazan (CTO at [Miranda NG](https: |
|
1,173
|
- [Positive Technologies](https://www.ptsecurity.com/) for funding and [The Standoff](https://standoff365.com/). |
|
1,174
|
|
|
1,175
|
Added features: |
|
1,176
|
|
|
1,177
|
- Provided package for [buildroot](https: |
|
1,178
|
- Binding for Nim is [available](https: |
|
1,179
|
- Added `mdbx_env_delete()` for deletion an environment files in a proper and multiprocess-safe way. |
|
1,180
|
- Added `mdbx_txn_commit_ex()` with collecting latency information. |
|
1,181
|
- Fast completion pure nested transactions. |
|
1,182
|
- Added `LIBMDBX_INLINE_API` macro and inline versions of some API functions. |
|
1,183
|
- Added `mdbx_cursor_copy()` function. |
|
1,184
|
- Extended tests for checking cursor tracking. |
|
1,185
|
- Added `MDBX_SET_LOWERBOUND` operation for `mdbx_cursor_get()`. |
|
1,186
|
|
|
1,187
|
Fixes: |
|
1,188
|
|
|
1,189
|
- Fixed missing installation of `mdbx.h++`. |
|
1,190
|
- Fixed use of obsolete `__noreturn`. |
|
1,191
|
- Fixed use of `yield` instruction on ARM if unsupported. |
|
1,192
|
- Added pthread workaround for buggy toolchain/cmake/buildroot. |
|
1,193
|
- Fixed use of `pthread_yield()` for non-GLIBC. |
|
1,194
|
- Fixed use of `RegGetValueA()` on Windows 2000/XP. |
|
1,195
|
- Fixed use of `GetTickCount64()` on Windows 2000/XP. |
|
1,196
|
- Fixed opening DB on a network shares (in the exclusive mode). |
|
1,197
|
- Fixed copy&paste typos. |
|
1,198
|
- Fixed minor false-positive GCC warning. |
|
1,199
|
- Added workaround for broken `DEFINE_ENUM_FLAG_OPERATORS` from Windows SDK. |
|
1,200
|
- Fixed cursor state after multimap/dupsort repeated deletes (https://libmdbx.dqdkfa.ru/dead-github/issues/121). |
|
1,201
|
- Added `SIGPIPE` suppression for internal thread during `mdbx_env_copy()`. |
|
1,202
|
- Fixed extra-rare `MDBX_KEY_EXIST` error during `mdbx_commit()` (https: |
|
1,203
|
- Fixed spilled pages checking (https: |
|
1,204
|
- Fixed `mdbx_load` for 'plain text' and without `-s name` cases (https://libmdbx.dqdkfa.ru/dead-github/issues/136). |
|
1,205
|
- Fixed save/restore/commit of cursors for nested transactions. |
|
1,206
|
- Fixed cursors state in rare/special cases (move next beyond end-of-data, after deletion and so on). |
|
1,207
|
- Added workaround for MSVC 19.28 (Visual Studio 16.8) (but may still hang during compilation). |
|
1,208
|
- Fixed paranoidal Clang C++ UB for bitwise operations with flags defined by enums. |
|
1,209
|
- Fixed large pages checking (for compatibility and to avoid false-positive errors from `mdbx_chk`). |
|
1,210
|
- Added workaround for Wine (https: |
|
1,211
|
- Fixed `ERROR_NOT_SUPPORTED` while opening DB by UNC pathnames (https: |
|
1,212
|
- Added handling `EXCEPTION_POSSIBLE_DEADLOCK` condition for Windows. |
|
1,213
|
|
|
1,214
|
|
|
1,215
|
## v0.9.1 2020-09-30 |
|
1,216
|
|
|
1,217
|
Added features: |
|
1,218
|
|
|
1,219
|
- Preliminary C++ API with support for C++17 polymorphic allocators. |
|
1,220
|
- [Online C++ API reference](https: |
|
1,221
|
- Quick reference for Insert/Update/Delete operations. |
|
1,222
|
- Explicit `MDBX_SYNC_DURABLE` to sync modes for API clarity. |
|
1,223
|
- Explicit `MDBX_ALLDUPS` and `MDBX_UPSERT` for API clarity. |
|
1,224
|
- Support for read transactions preparation (`MDBX_TXN_RDONLY_PREPARE` flag). |
|
1,225
|
- Support for cursor preparation/(pre)allocation and reusing (`mdbx_cursor_create()` and `mdbx_cursor_bind()` functions). |
|
1,226
|
- Support for checking database using specified meta-page (see `mdbx_chk -h`). |
|
1,227
|
- Support for turn to the specific meta-page after checking (see `mdbx_chk -h`). |
|
1,228
|
- Support for explicit reader threads (de)registration. |
|
1,229
|
- The `mdbx_txn_break()` function to explicitly mark a transaction as broken. |
|
1,230
|
- Improved handling of corrupted databases by `mdbx_chk` utility and `mdbx_walk_tree()` function. |
|
1,231
|
- Improved DB corruption detection by checking parent-page-txnid. |
|
1,232
|
- Improved opening large DB (> 4Gb) from 32-bit code. |
|
1,233
|
- Provided `pure-function` and `const-function` attributes to C API. |
|
1,234
|
- Support for user-settable context for transactions & cursors. |
|
1,235
|
- Revised API and documentation related to Handle-Slow-Readers callback feature. |
|
1,236
|
|
|
1,237
|
Deprecated functions and flags: |
|
1,238
|
|
|
1,239
|
- For clarity and API simplification the `MDBX_MAPASYNC` flag is deprecated. |
|
1,240
|
Just use `MDBX_SAFE_NOSYNC` or `MDBX_UTTERLY_NOSYNC` instead of it. |
|
1,241
|
- `MDBX_oom_func`, `mdbx_env_set_oomfunc()` and `mdbx_env_get_oomfunc()` |
|
1,242
|
replaced with `MDBX_hsr_func`, `mdbx_env_get_hsr` and `mdbx_env_get_hsr()`. |
|
1,243
|
|
|
1,244
|
Fixes: |
|
1,245
|
|
|
1,246
|
- Fix `mdbx_strerror()` for `MDBX_BUSY` error (no error description is returned). |
|
1,247
|
- Fix update internal meta-geo information in read-only mode (`EACCESS` or `EBADFD` error). |
|
1,248
|
- Fix `mdbx_page_get()` null-defer when DB corrupted (crash by `SIGSEGV`). |
|
1,249
|
- Fix `mdbx_env_open()` for re-opening after non-fatal errors (`mdbx_chk` unexpected failures). |
|
1,250
|
- Workaround for MSVC 19.27 `static_assert()` bug. |
|
1,251
|
- Doxygen descriptions and refinement. |
|
1,252
|
- Update Valgrind's suppressions. |
|
1,253
|
- Workaround to avoid infinite loop of 'nested' testcase on MIPS under QEMU. |
|
1,254
|
- Fix a lot of typos & spelling (Thanks to Josh Soref for PR). |
|
1,255
|
- Fix `getopt()` messages for Windows (Thanks to Andrey Sporaw for reporting). |
|
1,256
|
- Fix MSVC compiler version requirements (Thanks to Andrey Sporaw for reporting). |
|
1,257
|
- Workarounds for QEMU's bugs to run tests for cross-built[A library under QEMU. |
|
1,258
|
- Now C++ compiler optional for building by CMake. |
|
1,259
|
|
|
1,260
|
|
|
1,261
|
# |
|
1,262
|
|
|
1,263
|
Added features: |
|
1,264
|
|
|
1,265
|
- [Online C API reference](https: |
|
1,266
|
- Separated enums for environment, sub-databases, transactions, copying and data-update flags. |
|
1,267
|
|
|
1,268
|
Deprecated functions and flags: |
|
1,269
|
|
|
1,270
|
- Usage of custom comparators and the `mdbx_dbi_open_ex()` are deprecated, since such databases couldn't be checked by the `mdbx_chk` utility. |
|
1,271
|
Please use the value-to-key functions to provide keys that are compatible with the built-in libmdbx comparators. |
|
1,272
|
|
|
1,273
|
|
|
1,274
|
|
|
1,275
|
|
|
1,276
|
|
|
1,277
|
# |
|
1,278
|
|
|
1,279
|
- Added support multi-opening the same DB in a process with SysV locking (BSD). |
|
1,280
|
- Fixed warnings & minors for LCC compiler (E2K). |
|
1,281
|
- Enabled to simultaneously open the same database from processes with and without the `MDBX_WRITEMAP` option. |
|
1,282
|
- Added key-to-value, `mdbx_get_keycmp()` and `mdbx_get_datacmp()` functions (helpful to avoid using custom comparators). |
|
1,283
|
- Added `ENABLE_UBSAN` CMake option to enabling the UndefinedBehaviorSanitizer from GCC/CLANG. |
|
1,284
|
- Workaround for [CLANG bug](https: |
|
1,285
|
- Returning `MDBX_CORRUPTED` in case all meta-pages are weak and no other error. |
|
1,286
|
- Refined mode bits while auto-creating LCK-file. |
|
1,287
|
- Avoids unnecessary database file re-mapping in case geometry changed by another process(es). |
|
1,288
|
From the user's point of view, the `MDBX_UNABLE_EXTEND_MAPSIZE` error will now be returned less frequently and only when using the DB in the current process really requires it to be reopened. |
|
1,289
|
- Remapping on-the-fly and of the database file was implemented. |
|
1,290
|
Now remapping with a change of address is performed automatically if there are no dependent readers in the current process. |
|
1,291
|
|
|
1,292
|
|
|
1,293
|
# |
|
1,294
|
|
|
1,295
|
- Minor change versioning. The last number in the version now means the number of commits since last release/tag. |
|
1,296
|
- Provide ChangeLog file. |
|
1,297
|
- Fix for using libmdbx as a C-only sub-project with CMake. |
|
1,298
|
- Fix `mdbx_env_set_geometry()` for case it is called from an opened environment outside of a write transaction. |
|
1,299
|
- Add support for huge transactions and `MDBX_HUGE_TRANSACTIONS` build-option (default `OFF`). |
|
1,300
|
- Refine LTO (link time optimization) for clang. |
|
1,301
|
- Force enabling exceptions handling for MSVC (`/EHsc` option). |
|
1,302
|
|
|
1,303
|
|
|
1,304
|
# |
|
1,305
|
|
|
1,306
|
- Support for Android/Bionic. |
|
1,307
|
- Support for iOS. |
|
1,308
|
- Auto-handling `MDBX_NOSUBDIR` while opening for any existing database. |
|
1,309
|
- Engage github-actions to make release-assets. |
|
1,310
|
- Clarify API description. |
|
1,311
|
- Extended keygen-cases in stochastic test. |
|
1,312
|
- Fix fetching of first/lower key from LEAF2-page during page merge. |
|
1,313
|
- Fix missing comma in array of error messages. |
|
1,314
|
- Fix div-by-zero while copy-with-compaction for non-resizable environments. |
|
1,315
|
- Fixes & enhancements for custom-comparators. |
|
1,316
|
- Fix `MDBX_WITHOUT_MSVC_CRT` option and missing `ntdll.def`. |
|
1,317
|
- Fix `mdbx_env_close()` to work correctly called concurrently from several threads. |
|
1,318
|
- Fix null-deref in an ASAN-enabled builds while opening the environment with error and/or read-only. |
|
1,319
|
- Fix AddressSanitizer errors after closing the environment. |
|
1,320
|
- Fix/workaround to avoid GCC 10.x pedantic warnings. |
|
1,321
|
- Fix using `ENODATA` for FreeBSD. |
|
1,322
|
- Avoid invalidation of DBI-handle(s) when it just closes. |
|
1,323
|
- Avoid using `pwritev()` for single-writes (up to 10% speedup for some kernels & scenarios). |
|
1,324
|
- Avoiding `MDBX_UTTERLY_NOSYNC` as result of flags merge. |
|
1,325
|
- Add `mdbx_dbi_dupsort_depthmask()` function. |
|
1,326
|
- Add `MDBX_CP_FORCE_RESIZABLE` option. |
|
1,327
|
- Add deprecated `MDBX_MAP_RESIZED` for compatibility. |
|
1,328
|
- Add `MDBX_BUILD_TOOLS` option (default `ON`). |
|
1,329
|
- Refine `mdbx_dbi_open_ex()` to safe concurrently opening the same handle from different threads. |
|
1,330
|
- Truncate clk-file during environment closing. So a zero-length lck-file indicates that the environment was closed properly. |
|
1,331
|
- Refine `mdbx_update_gc()` for huge transactions with small sizes of database page. |
|
1,332
|
- Extends dump/load to support all MDBX attributes. |
|
1,333
|
- Avoid upsertion the same key-value data, fix related assertions. |
|
1,334
|
- Rework min/max length checking for keys & values. |
|
1,335
|
- Checking the order of keys on all pages during checking. |
|
1,336
|
- Support `CFLAGS_EXTRA` make-option for convenience. |
|
1,337
|
- Preserve the last txnid while copying with compactification. |
|
1,338
|
- Auto-reset running transaction in mdbx_txn_renew(). |
|
1,339
|
- Automatically abort errored transaction in mdbx_txn_commit(). |
|
1,340
|
- Auto-choose page size for large databases. |
|
1,341
|
- Rearrange source files, rework build, options-support by CMake. |
|
1,342
|
- Crutch for WSL1 (Windows subsystem for Linux). |
|
1,343
|
- Refine install/uninstall targets. |
|
1,344
|
- Support for Valgrind 3.14 and later. |
|
1,345
|
- Add check-analyzer check-ubsan check-asan check-leak targets to Makefile. |
|
1,346
|
- Minor fix/workaround to avoid UBSAN traps for `memcpy(ptr, NULL, 0)`. |
|
1,347
|
- Avoid some GCC-analyzer false-positive warnings. |
|
1,348
|
|
|
1,349
|
|
|
1,350
|
# |
|
1,351
|
|
|
1,352
|
- Workarounds for Wine (Windows compatibility layer for Linux). |
|
1,353
|
- `MDBX_MAP_RESIZED` renamed to `MDBX_UNABLE_EXTEND_MAPSIZE`. |
|
1,354
|
- Clarify API description, fix typos. |
|
1,355
|
- Speedup runtime checks in debug/checked builds. |
|
1,356
|
- Added checking for read/write transactions overlapping for the same thread, added `MDBX_TXN_OVERLAPPING` error and `MDBX_DBG_LEGACY_OVERLAP` option. |
|
1,357
|
- Added `mdbx_key_from_jsonInteger()`, `mdbx_key_from_double()`, `mdbx_key_from_float()`, `mdbx_key_from_int64()` and `mdbx_key_from_int32()` functions. See `mdbx.h` for description. |
|
1,358
|
- Fix compatibility (use zero for invalid DBI). |
|
1,359
|
- Refine/clarify error messages. |
|
1,360
|
- Avoids extra error messages "bad txn" from mdbx_chk when DB is corrupted. |
|
1,361
|
|
|
1,362
|
|
|
1,363
|
# |
|
1,364
|
|
|
1,365
|
- Fix `mdbx_load` utility for custom comparators. |
|
1,366
|
- Fix checks related to `MDBX_APPEND` flag inside `mdbx_cursor_put()`. |
|
1,367
|
- Refine/fix dbi_bind() internals. |
|
1,368
|
- Refine/fix handling `STATUS_CONFLICTING_ADDRESSES`. |
|
1,369
|
- Rework `MDBX_DBG_DUMP` option to avoid disk I/O performance degradation. |
|
1,370
|
- Add built-in help to test tool. |
|
1,371
|
- Fix `mdbx_env_set_geometry()` for large page size. |
|
1,372
|
- Fix env_set_geometry() for large pagesize. |
|
1,373
|
- Clarify API description & comments, fix typos. |
|
1,374
|
|
|
1,375
|
|
|
1,376
|
# |
|
1,377
|
|
|
1,378
|
- Fix returning MDBX_RESULT_TRUE from page_alloc(). |
|
1,379
|
- Fix false-positive ASAN issue. |
|
1,380
|
- Fix assertion for `MDBX_NOTLS` option. |
|
1,381
|
- Rework `MADV_DONTNEED` threshold. |
|
1,382
|
- Fix `mdbx_chk` utility for don't checking some numbers if walking on the B-tree was disabled. |
|
1,383
|
- Use page's mp_txnid for basic integrity checking. |
|
1,384
|
- Add `MDBX_FORCE_ASSERTIONS` built-time option. |
|
1,385
|
- Rework `MDBX_DBG_DUMP` to avoid performance degradation. |
|
1,386
|
- Rename `MDBX_NOSYNC` to `MDBX_SAFE_NOSYNC` for clarity. |
|
1,387
|
- Interpret `ERROR_ACCESS_DENIED` from `OpenProcess()` as 'process exists'. |
|
1,388
|
- Avoid using `FILE_FLAG_NO_BUFFERING` for compatibility with small database pages. |
|
1,389
|
- Added install section for CMake. |
|
1,390
|
|
|
1,391
|
|
|
1,392
|
# |
|
1,393
|
|
|
1,394
|
- Support for Mac OSX, FreeBSD, NetBSD, OpenBSD, DragonFly BSD, OpenSolaris, OpenIndiana (AIX and HP-UX pending). |
|
1,395
|
- Use bootid for decisions of rollback. |
|
1,396
|
- Counting retired pages and extended transaction info. |
|
1,397
|
- Add `MDBX_ACCEDE` flag for database opening. |
|
1,398
|
- Using OFD-locks and tracking for in-process multi-opening. |
|
1,399
|
- Hot backup into pipe. |
|
1,400
|
- Support for cmake & amalgamated sources. |
|
1,401
|
- Fastest internal sort implementation. |
|
1,402
|
- New internal dirty-list implementation with lazy sorting. |
|
1,403
|
- Support for lazy-sync-to-disk with polling. |
|
1,404
|
- Extended key length. |
|
1,405
|
- Last update transaction number for each sub-database. |
|
1,406
|
- Automatic read ahead enabling/disabling. |
|
1,407
|
- More auto-compactification. |
|
1,408
|
- Using -fsanitize=undefined and -Wpedantic options. |
|
1,409
|
- Rework page merging. |
|
1,410
|
- Nested transactions. |
|
1,411
|
- API description. |
|
1,412
|
- Checking for non-local filesystems to avoid DB corruption. |
|
1,413
|
|
|
1,414
|
|
|
1,415
|
|
|
1,416
|
For early changes see the git commit history. |
