Repository navigation
Conversation
Section.NewKey and File.NewSection scanned keyList and sectionList on every insert to find duplicates, which made loading quadratic in the number of keys per section (#377). Look them up in the maps that already exist instead. Loading 10,000 keys in one section drops from about 143 ms to 9 ms. inSlice has no callers left and is removed.
The lexer reads the whole input once and slices names, values and comments out of it, so a typical key line allocates nothing. One switch in lexer.next routes each line to a section, key, comment, raw body or nested value; readValue applies v1's value rules in v1's order, which its doc comment states as the contract. Behavior is unchanged with one exception: a Python-style multiline value is no longer cut at the reader buffer (4 KiB by default) and no longer fails with "key-value delimiter not found" (#210). Multi-line values are built in a strings.Builder instead of with += per line. parity_test.go loads every input with the released gopkg.in/ini.v1 and with this package and compares everything both stored, by reflection over the fields the two versions share. Its seeds cover testdata, the documented examples and the inputs from GitHub issues and pull requests; FuzzParseParity fuzzes inputs and options together. At 10,000 keys, Load allocates 4-5 times less than with the v1 parser.
Every section kept a second map that copied each value only to serve KeysHash, and growing it was the largest cost left when loading. Each Key now keeps that value in a hashValue field, updated where keysHash was: on creation, on overwrite in NewKey, and in SetValue. It can't reuse value, because Must* writes defaults into value but never changed KeysHash; the new test pins that. Loading 10,000 keys gets 25-33% faster.
Load and LoadSources take ...LoadOption instead of a LoadOptions struct, and LooseLoad, InsensitiveLoad and ShadowLoad become the Loose, Insensitive and AllowShadows options. Without options, parsing is exactly v1's Load. Every v1 option keeps its name and meaning. ReaderBufferSize is gone: the lexer has no read buffer to size. The writer settings that were package variables (LineBreak, DefaultHeader, PrettySection, PrettyFormat, PrettyEqual, DefaultFormatLeft/Right) and KeyValueDelimiterOnWrite are per-call WriteOptions of File.Write and File.SaveTo, so one caller can no longer change another's output. WriteToIndent and SaveToIndent become the Indent option; WriteTo stays for io.WriterTo. New options that v1 had no way to express: - DefaultSectionName renames the default section per file, e.g. AWS's [default] (#287), replacing the DefaultSection variable. - CommentSymbols sets the comment characters (#350). The default "#;" keeps v1 parsing; the writer uses them for comments it adds. Parse errors are wrapped in *ParseError with the 1-based line and column where the failing line's content starts (#341). Its message is "line L, column C: " followed by v1's message, and IsErrDelimiterNotFound and IsErrEmptyKeyName see through it. The parity tests map v1 options to v2 options and compare v1's error messages with the error inside ParseError.
IsBooleanType reports whether a key was read as a boolean key (#279). KeepEmptyShadowValues keeps empty values in ValueWithShadows and when writing shadow keys (#366, #342). In systemd units an empty assignment resets the values before it, which ValueWithShadows dropped since #316. Without the option the result is unchanged.
They follow encoding/json's shape (#337): Unmarshal maps INI data to a struct as MapTo does, and Marshal encodes a struct, or a pointer to one, as ReflectFrom and Write do.
The v2 module is served from unknwon.dev/ini/v2 instead of gopkg.in/ini.v2, so its versions come from v2.x.y tags rather than a gopkg.in branch. v1 stays at gopkg.in/ini.v1.
CI doesn't upload coverage to Codecov, so the config has no effect.
Go supports its latest two releases, 1.26 and 1.27; the module now requires the older of them instead of 1.22, which CI never tested.
- CommentSymbols compared the first byte of a line with the symbols'
bytes, so a key whose first character shared a lead byte with a
multi-byte symbol was read as a comment and lost. Symbols are now
compared as characters everywhere: line starts, SpaceBeforeInlineComment,
UnescapeValueCommentSymbols, and the comment prefix the writer adds,
which was invalid UTF-8.
- With CommentSymbols(""), the writer put ";" before comments, which
read back as keys. Writing a comment now returns an error instead.
- Marshal dropped all but one value of allowshadow fields and all but
one section of nonunique fields; it now reflects into a file that
allows both.
- KeepEmptyShadowValues still deduplicated repeated empty values, losing
every reset after the first. Empty values it keeps are never treated as
duplicates.
- A renamed default section (DefaultSectionName) was written as an empty
header when it had no keys. It is now skipped like the DEFAULT section;
with keys, its header is still written, as v1 did.
- Marshal and Unmarshal panicked on nil and on pointers to non-structs,
as ReflectFrom and MapTo did. They now return an error, and a nil
struct-pointer field writes no section.
Write now checks every key line with the lexer and picks a quoting that reads back to the same name and value, starting with v1's choice so ordinary output is unchanged. When no quoting works it returns an error instead of writing a file that loads differently. Also fixed in the writer: - Empty shadow values are written, so systemd-style resets survive. - A key written on several lines gets its comment before its last line, where the parser looks for it. - Nested values follow an empty value line, and boolean keys that also have values are written as they were read. - With Insensitive, an explicit [DEFAULT] section writes its own keys. - A default section with a comment keeps its header. - A raw body without a final line break no longer runs into the next section header. - Indent returns an error where an indented line would join the value before it. FuzzWriteRoundTrip and TestWriteRoundTripRandom load, write and reload files under random option sets and compare everything the File holds.
Reload parses every source into a fresh file and moves the result in, so keys and sections removed from a source are gone (#418) and shadows don't pile up (#273). Sections and keys that still exist keep their identity, so pointers callers hold see the new values. On error the file is unchanged. Append reads only the sources it adds, so changes made in memory stay and earlier sources aren't parsed twice. Readers are read once and kept, so a later Append or Reload doesn't fail on a used-up reader (#324).
Two keys referencing each other overflowed the stack in Key.String, a fatal error recover can't catch (#409). Expansion now tracks how deep references nest (99, as before per level), how many substitutions it makes (10,000) and how many bytes they produce (1 MiB), and stops at any limit, returning the value expanded so far.
- Key.Strings with an empty delimiter returned only after running out
of memory; it now returns the whole value.
- A negative index panicked in SectionWithIndex and corrupted the
section list in DeleteSectionWithIndex, which now returns an error.
SectionWithIndex("", n) no longer returns nil.
- With Insensitive, Section("") after deleting the default section
created a new section on every call; DeleteKey ignored the case.
- Section.Key("") returns a key that isn't added instead of nil.
- ChildSections listed non-unique sections once per occurrence.
- A default-section key referencing itself returns its raw value.
MapTo, StrictMapTo, Unmarshal, ReflectFrom and Marshal panicked on struct, named, pointer and nil types, nil targets, non-unique slices of pointers, and recursive struct types, and silently read or wrote wrong values in several cases. Values now go through one parser and one formatter, time.Time is recognised by its type, and recursive types are followed one level deep. Also fixed: overflowing numbers wrapped, StrictMapTo ignored invalid slice elements, zero and negative durations were dropped, *time.Time was written as a section, sub-second times were truncated, slice elements with the delimiter or a trailing backslash didn't read back, nil pointers were written as empty keys, a nonunique or reflector field stopped later fields from being written, allowshadow lost the comment and dropped values, uint8 didn't map, embedded pointers were replaced, and a nonunique field without sections failed the whole MapTo. The README lists the changes valid input can show.
With BlockMode, reads of keys, values, sections and bodies, and all of Write, took no lock, and Append parsed into the shared file, so reading while another goroutine reloaded, appended, set values or reflected was a data race. Public reads now take the read lock, mutations the write lock, and the parser and internal code use lock-free helpers, so Load and Reload parse a private file and Append holds the lock. Reload no longer rewrites the fields readers use to find the lock. Loading is slightly faster; a locked read costs one RLock pair. FuzzStructMapping loads arbitrary input under random options, calls every accessor, maps it, and checks that Unmarshal reads back what Marshal writes. It found that allowshadow slices lost repeated, empty and delimiter-containing elements, and that nonunique sections were merged on reading. Marshal now keeps repeated shadow values and escapes elements; Unmarshal enables the options allowshadow and nonunique fields need, and splits shadow values the way Key.Strings does.
A key with nested values that later became a boolean key was written with the boolean line first, and the parser can't add nested values to a boolean key, so the output failed to load. The nested values now go before the boolean line, after an empty value.
Renames: SnakeCase, SectionNames, KeyNames, KeyValues, Values,
IsBoolean, CaseInsensitive*, IgnoreMissingFiles, FirstSourceOnly,
RawSections and WriteDelimiter. Section.Haskey is removed. File.BlockMode
becomes the NoLocking load option, so locking can't be switched off
while the file is in use. The error structs and IsErr* functions become
sentinel errors for errors.Is, with the same messages. The *WithMapper
functions become a NameMapping load option, and the package-level
ReflectFrom functions are dropped in favor of File.ReflectFrom.
interface{} is any throughout.
The In*, Range*, typed list, Valid* and Strict* list methods of Key,
37 in all, become ini.In, ini.Range and ini.List[T]. They parse with
the struct mapping parser, so every Scalar type works, including sized
and named types, with overflow checks. The eight list parsers and the
exported Parser type that boxed every element are gone.
Internal helpers are named for what they do: decodeValue, decodeSlice,
decodeStruct, decodeSections, encodeValue, encodeSlice, encodeStruct,
newDataSource, loadSource and expandValue.
List returns the parsed elements with the zero value for one that doesn't parse, as v1's Ints did, so it needs no error handling. ValidList and StrictList bring back ValidInts and StrictInts, for any Scalar type. TimesFormat, ValidTimesFormat, StrictTimesFormat, InTimeFormat and RangeTimeFormat stay on Key, as a layout can't be a type parameter. Range also takes time.Time, replacing RangeTime, and In compares times as instants with Equal, as v1 did, instead of ==, which failed for the same instant in another zone.
Section and Key comments become Comment() and SetComment(), which take the file's lock like every other accessor. NameMapper and ValueMapper move from embedded File fields to the NameMapping and ValueMapping load options, so they can't change while other goroutines read values. A File keeps its sections in one ordered slice plus a name index, in place of sectionList, sectionIndexes and the map. A Section keeps its keys the same way, in place of keyList. Shadow values are a []string on the key instead of a *Key each, which drops the isShadow flag. KeyValues now reports the value each key holds, including defaults stored by Must* methods; v1's KeysHash left those out. Loading 10,000 keys uses 7-15% less memory and is faster in every case.
Gogs calls Section.ReflectFrom(&I18n) with I18n a package-level *i18nConf. v1 dereferenced the extra pointer; v2 had started returning "not a pointer to a struct".
PrettyFormat(bool) becomes AlignDelimiters(bool) and PrettySection(bool)
becomes BlankLineBetweenSections(bool), naming the output literally as
encoding/json/jsontext does with SpaceAfterColon and Multiline.
PrettyEqual() is removed: DelimiterSpaces(" ", " ") writes the same
bytes. PreserveSurroundedQuote() becomes PreserveSurroundingQuotes(),
fixing the grammar and keeping the verb jsontext uses in
PreserveRawStrings.
AllowShadows(), CaseInsensitive(), DefaultHeader() and the other 18 boolean options become AllowShadows(true) and so on, matching AlignDelimiters(bool) and BlankLineBetweenSections(bool). An option can now be set from a config flag, as in AllowShadows(cfg.Shadows), and a later option can turn off an earlier one. KeepEmptyShadowValues becomes PreserveEmptyShadowValues, matching PreserveSurroundingQuotes.
Locking(false) reads without a double negative and maps directly from v1's File.BlockMode = false. Unmarshal adds the options its allowshadow and nonunique tags need after the caller's, so AllowShadows(false) or a config flag that defaults to false can't silently drop slice elements and sections.
api_parity_test.go calls v1 and v2 on the same files and values: every Key accessor, the lenient, Valid and Strict lists, In, Range and Must*, Write, MapTo and StrictMapTo into a struct with every field kind v1 maps, ReflectFrom of the mapped struct, and the struct API with targets of every shape (T, *T, **T, nil, slices). Where v1 doesn't panic, v2 must return the same, except for the deliberate changes in the README. Planting the old ReflectFrom(&ptr) bug back makes it fail. It found that allowshadow slices un-escaped backslashes, which v1 never did: a value such as C:\\dir mapped to a different string. They are read as in v1 again, each value split at the delimiter, and Marshal and ReflectFrom return an error for an element that contains the delimiter, which v1 wrote and read back as two elements.
v1's Must* methods wrote the default into the key, so later reads, MapTo and Write saw it, and saving a config wrote out every default the program had asked for. They now only read; call SetValue to store a value. The API parity test resets v1's value after each Must* call. Without the write-back it also exposed two deliberate MapTo differences that the old chained defaults hid (zero durations, out-of-range list elements), now listed with the others.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is v2 of the library, at the new module path
unknwon.dev/ini/v2. It has a new single-pass lexer, a functional options API, no package-level globals, and loading that takes linear time. With no options, v2 parses every input exactly as v1 does. A differential test suite checks this against the releasedgopkg.in/ini.v1v1.67.3.Speed
BenchmarkLoad10kKeys, 10,000 keys, Go 1.26, median of 3 runs:""", continuation,-, CRLF)Section.NewKeyandFile.NewSectionscanned the whole key or section list on every insert to find duplicates. They now look the name up in the maps that already exist.Filekeeps its sections in one ordered slice plus a name index, and aSectionits keys the same way, replacing v1's parallel name lists and index slices. Shadow values are a[]stringon the key instead of a whole*Keyeach, and the secondkeysHashmap is gone. This cut another 7–15% of memory per 10,000-key load.Parity with v1
parity_test.goloads the same input twice into oneFile, with v1 and with v2. It compares everything both stored, reading v1's unexported fields by reflection and v2's directly, so it covers state the public API doesn't expose:ParseError)It covers:
testdata/*.inifile, the examples from the docs site and GoDoc, and inputs taken from 155 parser-related issues and PRs (420 scanned).FuzzParseParity: fuzzes the input together with 20 option bits. A final 3-minute run made 3.31 million comparisons with no difference, and earlier runs during development made about 11 million more.TestParseParityRandom: 50,000 generated documents built from fragments that sit on feature boundaries, plus every seed under 500 random option sets.I also planted behavior changes in the lexer on purpose; the parity tests caught every one that changed behavior.
The public API is compared the same way.
api_parity_test.gocalls v1 and v2 on the same files and values. It covers:Keyaccessor: the lenient,ValidandStrictlists,In,RangeandMust*;Write;MapToandStrictMapTointo a struct with every field kind v1 maps, andReflectFromof the result;T,*T,**T, nil and slices.Where v1 doesn't panic, v2 must return the same, except for the deliberate changes listed in the README.
TestAPIParity,TestAPIParityShapesandFuzzAPIParityrun it. It found thatallowshadowslices un-escaped backslashes, which v1 never did, and that is fixed. Planting the earlierReflectFrom(&ptr)regression back makes it fail.The one deliberate parsing difference (go-ini/ini PR 210): v1 read a Python-style multiline value only as far as one read buffer (4 KiB by default). A longer value was cut mid-line and then failed with
key-value delimiter not found. v2 reads the whole value.TestParsePythonMultilineLongerThanBuffercovers this.API changes
Loading
Load(source, opts ...LoadOption)andLoadSources([]any{...}, opts...)take functional options; theLoadOptionsstruct is gone.LooseLoad,InsensitiveLoadandShadowLoadbecome theIgnoreMissingFiles(true),CaseInsensitive(true)andAllowShadows(true)options.bool, as inencoding/json/v2andjsontext, so it can be set from a config flag (ini.AllowShadows(cfg.Shadows)) and a later option can turn off an earlier one.DebugFuncbecomesDebug(fn).ReaderBufferSizeis removed, because the lexer has no read buffer to size.Writing
LineBreak,DefaultHeader,PrettySection,PrettyFormat,PrettyEqualandDefaultFormatLeft/Right, plus theKeyValueDelimiterOnWriteload option, become per-callWriteOptions ofFile.WriteandFile.SaveTo. One caller can no longer change another caller's output. The defaults produce the same output as v1.WriteToIndentandSaveToIndentbecome theIndentoption.PrettyFormat→AlignDelimiters(bool),PrettySection→BlankLineBetweenSections(bool).PrettyEqualis gone;DelimiterSpaces(" ", " ")writes the same bytes. The names followencoding/json/jsontext(SpaceAfterColon,Multiline).WriteTostays, forio.WriterTo.Default section:
DefaultSectionis now a constant.DefaultSectionName("default")renames the default section per file, for example for AWS config files (Allow for lower case [default] section #287).Removed: the deprecated
DEFAULT_SECTION,AllCapsUnderscoreandSection.Haskey.Clearer names:
SnackCase→SnakeCase.SectionStrings/KeyStrings/KeysHash→SectionNames/KeyNames/KeyValues.ValueWithShadows→Values,IsBooleanType→IsBoolean.Insensitive*→CaseInsensitive*,Loose→IgnoreMissingFiles,ShortCircuit→FirstSourceOnly,UnparseableSections→RawSections,PreserveSurroundedQuote→PreserveSurroundingQuotes,KeyValueDelimiterOnWrite→WriteDelimiter.File.BlockModefield becomes theLocking(bool)load option (Locking(false)for v1'sBlockMode = false), so locking can't be switched off while the file is in use.No unlocked public fields:
Section.CommentandKey.CommentbecomeComment()andSetComment(c), which take the file's lock.File.NameMapperandFile.ValueMapperfields become theNameMapping(m)andValueMapping(fn)load options, so they can't change while other goroutines read values.Must*only read.k.MustInt(3)returns 3 for a missing or invalid value but no longer stores it in the key. In v1 the default showed up in later reads and inWriteoutput, so saving a config wrote out every default the program had asked for. Code that reads a default back later callsk.SetValue. Grafana's whole parsed config and Gogs' output are unchanged; Gitea relies on the write-back in the 45 places it marksINI-MUST-SIDE-EFFECTand needsSetValuethere.Errors: the
ErrDelimiterNotFound/ErrEmptyKeyNamestructs and theirIsErr*functions become sentinel errors checked witherrors.Is. Messages are unchanged.Fewer entry points:
MapToWithMapperandStrictMapToWithMapperbecome aNameMapping(m)load option. The package-levelReflectFromandReflectFromWithMapperare dropped in favor ofFile.ReflectFrom.Generic accessors: 34 typed
Keymethods become generic functions with the same behavior:Ints,ValidInts,StrictIntsand the same for the other types →ini.List[T],ini.ValidList[T]andini.StrictList[T].In*methods →ini.In(k, def, candidates).Range*methods →ini.Range(k, def, lo, hi), which also takestime.Time.They parse with the same code as struct mapping, so every
Scalartype works, includingint8,time.Durationand named types. Internally, the eight list parsers and the exportedParsertype that boxed every element are gone.String,Int,Bool,Duration,Strings, theMust*methods and the custom-layout time methods (TimesFormat,InTimeFormat,RangeTimeFormatand friends) stay onKey.Import path:
unknwon.dev/ini/v2; v1 stays atgopkg.in/ini.v1. v2 versions come fromv2.x.ytags.Requires Go 1.26, the oldest supported Go release; CI tests 1.26 and 1.27.
The README has a migration table.
New features (all opt-in; defaults unchanged)
*ParseErrorwith a 1-basedLineandColumn, so a message readsline 4, column 3: key-value delimiter not found: foo.errors.Is(err, ini.ErrDelimiterNotFound)anderrors.Is(err, ini.ErrEmptyKeyName)see through it.CommentSymbols(s)sets them. The default"#;"keeps v1 behavior, and the writer uses the configured symbols for comments it adds.PreserveEmptyShadowValues(true)keeps empty values inValues, for example for systemd units, where an empty assignment resets earlier ones.Key.IsBoolean()(Export the Key.isBooleanType field or provide a method #279).MarshalandUnmarshal(Marshal and unmarshal functions #337), shaped likeencoding/json.Hardening
These fix v1 bugs outside the parser, so parse parity is unaffected.
Writeoutput loads back to the same file. Each key line is checked with the lexer before it is written. v1's quoting is tried first, so ordinary output is byte-identical to v1 (the golden test passes unchanged). If v1's quoting reads back differently, other quotings are tried. If none works,Writereturns an error instead of writing a corrupt file. Fixed along the way:\, containing""", or starting with a delimiterInsensitive, an explicit[DEFAULT]section was written with the wrong section's keysIndenttogether with Python multiline or nested values indented lines that then joined the previous valueFuzzWriteRoundTripandTestWriteRoundTripRandomload, write and reload under random option sets and compare everything theFileholds. The final 3-minute fuzz run made 1.38M executions with no failure. Write speed is unchanged: 8.8 ms vs 10.2 ms before for 10k keys, same allocations.Reloadreplaces the contents (Reload retains keys removed from the source file #418, ShadowLoad和Reload问题 #273). Keys and sections removed from the sources are gone, and shadows no longer pile up.*Sectionand*Keypointers held across a reload see the new values. If loading fails, the file is unchanged.Appendreads only the new sources (Append() get file already closed when trying to append a second io.Reader #324). In-memory changes survive anAppend.io.Readersources are read once and kept, so a laterAppendorReloadno longer fails on an exhausted reader.%(name)sexpansion stops (Key.String()overflows the stack when two keys reference each other #409). Keys that reference each other overflowed the stack inKey.String, a fatal error thatrecovercan't catch. Expansion is now limited by nesting depth, substitution count and total output size.API misuse no longer panics or hangs. These calls now behave:
Key.Strings("")Section.Key("")Section("")after deleting the default section withInsensitiveDeleteKeywithInsensitiveChildSectionswith non-unique namesStruct mapping returns errors instead of panicking.
MapTo,StrictMapTo,Unmarshal,ReflectFromandMarshalhandle these types:Silent wrong values are fixed too: overflowing numbers, invalid slice elements in strict mode, zero and negative durations,
*time.Time, sub-second times, unescaped slice elements, nil pointers,allowshadowcomments, and uint8. The README lists every difference valid input can show.Concurrent use is race-free. v1 locked only some writes, even with
BlockModeon (the default). Reading a key while another goroutine calledReload,Append,SetValueorWritewas a data race. Public reads now take the read lock and every mutation takes the write lock. The parser uses lock-free helpers on a file no one else can see yet, so loading is slightly faster.TestFile_Concurrencyruns readers alongside every kind of writer under-race: the previous code had 40 races in it and the new code has none. A locked read costs about 18 ns more.Unmarshalreads back whatMarshalwrites. A newFuzzStructMappingtest loads arbitrary input, calls every accessor, maps it into a struct with every supported field kind, and checks the round trip. It found thatallowshadowslices lost repeated and empty elements (an element containing the delimiter now makesMarshalreturn an error, as v1 split it), and thatnonuniquesections were merged when read back. Both are fixed.Not in this PR
The v1 behaviors that users report as bugs are intentionally unchanged, so parity holds. They include
#inside quotes (#393), even numbers of trailing backslashes (#347), comments at the end of the file (#362), and blank lines in Python multiline values (#205). Each one would be an opt-in option or a v2-default decision in a follow-up, and the parity suite shows exactly what each change affects.