| Age | Commit message (Collapse) | Author |
|---|
|
Our http method encodes the URI again which results in the double
encoding we have unwrap in the webserver (we did already, but we skip
the filename handling now which does the first decode).
|
|
|
|
Unroll pkgCache::sHash 8 times and break up the dependency between
the iterations by expanding the calculation
H(n) = 33 * H(n-1) + c
8 times rather than performing it 8 times. This seems to yield about
a 0.4% performance improvement.
I tried unrolling 4 and 2 bytes as well, those only having 3 ifs at
the end rather than 1 small loop; but that was actually slower -
potentially the code got to large and the cache went bonkers.
I also tried unrolling 4 times instead of 8, thinking that smaller
code might yield better results overall then, but that was slower as
well.
|
|
|
|
XXH3 is faster than both our CRC32c implementation as well
as DJB hash for hash table hashing, so meh, let's switch to
it.
|
|
|
|
|
|
We now have over 100k package names, my Ubuntu system has 125k
arleady, so increase the hash table size to match, this will cost
us about a MB in cache size, but give a very nice speed up somewhere
around 3%-4% or so.
|
|
|
|
The integer overflow was detected by DonKult who added a check like this:
(std::numeric_limits<decltype(Itm.Size)>::max() - (2 * sizeof(Block)))
Which deals with the code as is, but also still is a fairly big limit,
and could become fragile if we change the code. Let's limit our file
sizes to 128 GiB, which should be sufficient for everyone.
Original comment by DonKult:
The code assumes that it can add sizeof(Block)-1 to the size of the item
later on, but if we are close to a 64bit overflow this is not possible.
Fixing this seems too complex compared to just ensuring there is enough
room left given that we will have a lot more problems the moment we will
be acting on files that large as if the item is that large, the (valid)
tar including it probably doesn't fit in 64bit either.
|
|
Like the code in arfile.cc, MemControlExtract also has buffer
overflows, in code allocating memory for parsing control files.
Specify an upper limit of 64 MiB for control files to both protect
against the Size overflowing (we allocate Size + 2 bytes), and
protect a bit against control files consisting only of zeroes.
|
|
Tarballs have long names and long link targets structured by a
special tar header with a GNU extension followed by the actual
content (padded to 512 bytes). Essentially, think of a name as
a special kind of file.
The limit of a file size in a header is 12 bytes, aka 10**12
or 1 TB. While this works OK-ish for file content that we stream
to extractors, we need to copy file names into memory, and this
opens us up to an OOM DoS attack.
Limit the file name size to 1 MiB, as libarchive does, to make
things safer.
|
|
GHSL-2020-169: This first hunk adds a check that we have more files
left to read in the file than the size of the member, ensuring that
(a) the number is not negative, which caused the crash here and (b)
ensures that we similarly avoid other issues with trying to read too
much data.
GHSL-2020-168: Long file names are encoded by a special marker in
the filename and then the real filename is part of what is normally
the data. We did not check that the length of the file name is within
the length of the member, which means that we got a overflow later
when subtracting the length from the member size to get the remaining
member size.
The file createdeb-lp1899193.cc was provided by GitHub Security Lab
and reformatted using apt coding style for inclusion in the test
case, both of these issues have an automated test case in
test/integration/test-ubuntu-bug-1899193-security-issues.
LP: #1899193
|
|
This allows patterns like ~nalpha~nbeta and ~nalpha!~nbeta to
work like they do in APT.
Also add a comment to remind readers that everything in START
should be in short too.
Cc: stable >= 2.0
|
|
The compiler does not know that the size is small and thinks we might
be doing a stack buffer overflow of the vla:
Add APT_ASSUME macro and silence -Wstringop-overflow in HexDigest()
The compiler does not know that the size of a hash is at most 512 bit,
so tell it that it is.
../apt-pkg/contrib/hashes.cc: In function ‘std::string HexDigest(gcry_md_hd_t, int)’:
../apt-pkg/contrib/hashes.cc:415:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
415 | Result[(Size)*2] = 0;
| ~~~~~~~~~~~~~~~~~^~~
../apt-pkg/contrib/hashes.cc:414:9: note: at offset [-9223372036854775808, 9223372036854775807] to an object with size at most 4294967295 declared here
414 | char Result[((Size)*2) + 1];
| ^~~~~~
Fix this by adding a simple assertion. This generates an extra two
instructions in the normal code path, so it's not exactly super costly.
|
|
Fixes lookup in as-installed testing
Gbp-Dch: ignore
|
|
|
|
This is more accurate
|
|
Apply hints suggested by the multi-arch hinter
See merge request apt-team/apt!137
|
|
Fix typo in Catalan translation.
See merge request apt-team/apt!132
|
|
Enhance rred for possible external usage
See merge request apt-team/apt!136
|
|
+ apt-doc, libapt-pkg-doc: Add Multi-Arch: foreign.
Changes-By: apply-multiarch-hints
|
|
|
|
|
|
The acquire system mode does this for a long time already and as it is
easy to implement and handy for manual testing as well we can support
it in the other modes, too.
|
|
Merging patches is a bit of non-trivial code we have for client-side
work, but as we support also server-side merging we can export this
functionality so that server software can reuse it.
Note that this just cleans up and makes rred behave a bit more like all
our other binaries by supporting setting configuration at runtime and
supporting --help and --version. If you can make due without this, the
now advertised functionality is provided already in earlier versions.
|
|
In LP#835625, it was reported that apt did not unpack multi-arch
packages in the correct order, and dpkg did not like that. The fix
also made apt configure packages together, which is not strictly
necessary.
This turned out to cause issues now, because of dependencies on
libc6:i386 that caused immediate configuration of that to not
work.
Work around the issue by not configuring multi-arch: same packages
in lockstep if they have the immediate flag set. This will be the
pseudo-essential set, and given how essential works, we mostly need
the native arch to work correctly anyway.
LP: #1871268
Regression-Of: 30426f4822516bdd26528aa2e6d8d69c1291c8d3
|
|
|
|
That mostly means deleting symbols which went private or have
disappeared and were previously compiler artefacts.
|
|
The versions "needing" these fixes are at least five years old, so in an
effort to save massive amounts of runtime and disk space (on aggregate at
least) we can drop these lines.
Reported-By: lintian maintainer-script-supports-ancient-package-version
|
|
Reported-By: dh_missing
|
|
| CMake Warning (dev) at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
| The package name passed to `find_package_handle_standard_args` (Berkeley)
| does not match the name of the calling package (BerkeleyDB). This can lead
| to problems in calling code that expects `find_package` result variables
| (e.g., `_FOUND`) to follow a certain pattern.
| Call Stack (most recent call first):
| CMake/FindBerkeleyDB.cmake:57 (find_package_handle_standard_args)
| CMakeLists.txt:83 (find_package)
| This warning is for project developers. Use -Wno-dev to suppress it.
And indeed, we checked for BERKLEY_DB_FOUND which was not defined so our
HAVE_BDB was not set – just that it is never used, so it wasn't noticed.
|
|
Closes: #968414
|
|
mirror.fail points to porn now apparently.
Cc: stable
|
|
We accidentally excluded virtual packages by excluding every
group that had a package, but where the package had no versions.
Rewrite the code so the lookup consistently uses VersionList()
instead of FirstVersion and FindPkg("any") - those are all the
same, and this is easier to read.
|
|
We passed "false" instead of false, and that apparently got
cast to bool, because it's a non-null pointer.
LP: #1876495
|
|
|
|
We are seeing more and more installations fail due to immediate
configuration issues related to libc6. Immediate configuration is
supposed to ensure that an essential package is configured immediately,
just in case some other packages use a part of the essential package
that only works if that package is configured.
This used to be a warning, it was turned into an error in some commit I
can't remember right now, but importantly, the error missed a return,
which means that ordering completed succesfully and packages were being
installed anyway; and after all that happened successfully, we'd print
an error at the end and exit with an error code, which is not super
useful.
Revert the error back to a warning such that the behavior stays the same
but we do not fail (unless we mess up ordering which then gets caught by
a consistency check later on.
Closes: #953260
Closes: #972552
LP: #1871268
|
|
Closes: #970037
[jak: Fix typo extended_status -> extended_states]
|
|
|
|
|
|
Closes: #969086
|
|
|
|
Add better acquire debugging support
See merge request apt-team/apt!130
|
|
The old code was fairly confusing, and contradictory. Notably, the
second `if` also only applied to the Data state, whereas we already
terminated the Data state earlier. This was bad.
The else fallback applied in three cases:
(1) We reached our limit
(2) We are Persistent
(3) We are headers
Now, it always failed as a transient error if it had
nothing left in the buffer. BUT: Nothing left in the buffer
is the correct thing to happen if we were fetching content.
Checking all combinations for the flags, we can compare the results
of Die() between 2.1.7 - the last "known-acceptable-ish" version
and this version:
2.1.7 this
Data !Persist !Space !Limit OK (A) OK
Data !Persist !Space Limit OK (A) OK
Data !Persist Space !Limit OK (C) OK
Data !Persist Space Limit OK OK
Data Persist !Space !Limit ERR ERR *
Data Persist !Space Limit OK (B) OK
Data Persist Space !Limit ERR ERR
Data Persist Space Limit OK OK
=> Data connections are OK if they have not reached their limit,
or are persistent (in which case they'll probably be chunked)
Header !Persist !Space !Limit ERR ERR
Header !Persist !Space Limit ERR ERR
Header !Persist Space !Limit OK OK
Header !Persist Space Limit OK OK
Header Persist !Space !Limit ERR ERR
Header Persist !Space Limit ERR ERR
Header Persist Space !Limit OK OK
Header Persist Space Limit OK OK
=> Common scheme here is that header connections are fine if they have
read something into the input buffer (Space). The rest does not matter.
(A) Non-persistent connections with !space always enter the else clause, hence success
(B) no Space means we enter the if/else, we go with else because IsLimit(), and we succeed because we don't have space
(C) Having space we do enter the while (WriteSpace()) loop, but we never reach IsLimit(),
hence we fall through. Given that our connection is not persistent, we fall through to the
else case, and there we win because we have data left to write.
|
|
We do not want to end up in a code path while reading content
from the server where we have local data left to write, which
can happen if a previous read included both headers and content.
Restructure Flush() to accept a new argument to allow incomplete
flushs (which do not match our limit), so that it can flush as
far as possible, and modify Go() and use that before and after
reading from the server.
|
|
This causes some more issues, really.
|
|
We have successfully finished reading data if our buffer is empty,
so we don't need to do any further checks.
|
|
If we have errors pending, always log them with our failure
message to provide more context.
|
|
Default Acquire::AllowReleaseInfoChange::Suite to "true"
See merge request apt-team/apt!128
|
