Tag: pkgstats

pkgstats is a tool that gathers and analyses installed packages of Arch Linux users. It started as a small shell script back in 2008 and helps us among other things to determine which packages are no longer used but also which packages from the AUR are popular candidates.

Previously I rewrote the server part and added historical statistics per package and the ability to compare the popularity of several packages over time. It also introduced a public API which is now used by the pkgstats client.

The new pkgstats client

Today I released the initial version of pkgstats v3, the client that can be installed via pacman. While in the past it was only used to send usage data to the server, the cli has now an interface to search for package statistics and compare different packages.

Search for packages

You can now search for packages by using pkgstats search <query>

~> pkgstats search firefox
firefox                             77.57
firefox-developer-edition           7.42
firefox-i18n-en-us                  3.76
firefox-i18n-de                     3.60
firefox-i18n-fr                     2.08
firefox-ublock-origin               1.93
firefox-i18n-en-gb                  1.90
firefox-nightly                     1.58
firefox-adblock-plus                1.51
firefox-extension-https-everywhere  1.32

10 of 288 results

See more results at https://pkgstats.archlinux.de/packages#query=firefox

Show and compare packages

You can lookup the popularity of a single package or compare a number of packags at once. Simply type pkgstats show <package> [<package>...]

~> pkgstats show nodejs go php python rust
python         99.73
nodejs         65.26
go             61.34
php            33.10
rust           20.49

See more results at https://pkgstats.archlinux.de/compare/packages#packages=go,nodejs,php,python,rust

Direct links to web graphs

As you saw in the examples above, pkgstats now also prints an URL representing your query. Following these link renders a visual representation of historical data.

Searching for packages

Comparing packages over time

Submit your package list

As with previous versions of pkgstats a list of your installed packages can be submitted to the server. This is done on a weekly basis triggered by a systemd timer. If you like to inspect what data would be transmitted you can print the JSON output via pkgstats submit --dump-json

~> pkgstats submit --dump-json 
{
  "version": "3",
  "system": {
    "architecture": "x86_64"
  },
  "os": {
    "architecture": "x86_64"
  },
  "pacman": {
    "mirror": "https://mirror.pkgbuild.com/",
    "packages": [
      "a52dec",
      "aalib",
      "accountsservice",
      "acl",
       ...
      "zsh",
      "zstd",
      "zvbi",
      "zxing-cpp"
    ]
  }
}

Integrated help and shell completion

The new client comes with an integrated help command for each sub command and flag. The package also ships with shell completions for Bash, Zsh and Fish.

Feedback welcome

I rewrote the initial shell script in Go and released it as version 2.5. Version 3 is a continuation of this effort and represents just the beginning. If you have any suggestions to improve pkgstats or ideas for interesting and useful features please let me know.

Further information

You’ll find more information on the pkgstats home page and GitHub.

The first 64 Bit x86 CPU was released ten years ago: the AMD Opteron. Intel followed with compatible processors about one year later. Another two years later, in 2006, a first port of Arch Linux to the x86_64 platform was available. Starting outside the official repositories, the new architecture was merged into the Arch project and became a first class citizen within the following years. Right from the start it was designed as a pure 64 Bit system and kept in sync with the i686 tree. That also meant no multilib support at first and every package was compiled for x86_64.

Of course getting there was not that easy. Our repositories and tools like pacman did not have any concept of different architectures. While most software would just work fine when compiled for x86_64, we occasionally ran into issues that were specific to the 64 Bit architecture.

Years later the relation between 32 and 64 bit architecture seem to have switched roles.  Developers have chosen x86_64 as their primary platform, our i686 packages seem to get tested by less users and we see issues specific to the 32 Bit architecture. We recently relaxed our signoff policy for i686 packages in order to counteract this trend and avoid holding back development due to lack of testing.

To back these observations with some numbers, I analyzed the architecture submitted by users via pkgstats over the past few years. Even if we assume that these data are not absolutely accurate and I could not find all the data gathered between 2009 and 2010, the results are clear: While the x86_64 version of Arch Linux was installed by only 20% of our users in 2008, its usage grew to 80% in 2013. It is fair to say that within the next years i686 usage will fade away.

For a few month we also collect data about the actual CPU architecture users have. Right now about 93% operate a CPU that would be capable of running a 64 Bit system. There is not much data yet, but it seems to be a slowly rising number.

We are pretty close to the point where i686 Arch Linux installations have been almost completely replaced by x86_64. This of course means that it might no longer be practical to provide support for an architecture that is barely used. The question is not if 32 bit support might be dropped,  but when.

Don’t Panic! I don’t see us dropping i686 support this year, but I would not dare to predict anything beyond that date. As always these decision are not based on business or politics. We will see when i686 usage will drop below a critical number and developers loose interest in packaging and supporting that architecture.