Blogger still pushing Google+

U.S. DoE unvails $500m supercomputer

From the New York Times: DoE is unveiling a new $500m exascale supercomputer, produced by a collaboration between Intel and Cray. This is part of the computing arms race with China, which has been making big strides in supercomputing in the past decade.

The supercomputer, called Aurora, is a retooling of a development effort first announced in 2015 and is scheduled to be delivered to the Argonne National Laboratory near Chicago in 2021. Lab officials predict it will be the first American machine to reach a milestone called “exascale” performance, surpassing a quintillion calculations per second.

Annoyance with Google+

A few years ago, I migrated Blogger comments here to Google+ comments, so that comments would be linked to commenters' Google+ profiles, and appear in their Google+ feeds. That, unhelpfully, deleted all existing old comments.

Now, the reverse is happening. Since Google+ is going away, all Google+ comments will disappear with it, with no attempt being made to port existing comments to Blogger comments.

Facebook's new suite of open source Linux kernel components and tools

Facebook has just announced a bunch of useful Linux kernel components and tools; specifically, useful for shared servers, which may include HPC servers. I could see oomd and cgroup2, in particular, being useful. Oomd takes out-of-memory handling into userspace and tries to take corrective action before an OOM occurs in kernel. Cgroup2 seems to be an successor to cgroups, which allows controlling the amount of system resources assigned to groups of workloads.

XFS group and project quotas

XFS supports a quota system, that can handle quotas by user, group, or project. The project quota system is meant for setting quotas on directory hierarchies, e.g. you want to set quotas on users' home directories, but allow them a different quota in some shared directory.

What the documentation and the man page for xfs_quota(8) do not mention is that group quotas and project quotas are mutually exclusive. I.e. if you turn on group quotas on filesystem, you cannot turn on project quotas, and vice versa.

Here are my simple scripts for setting up XFS project quotas to set quotas on users' home directories: https://github.com/prehensilecode/xfs-project-quotas

Docker containers for high performance computing

I have started to see more science application authors/groups provide their applications as Docker images. Having only a passing acquaintance with Docker (and with containers in general), I found this article at The New Stack useful: Containers for High Performance Computing (Joab Jackson).

The vendors of the tools I use -- Bright Cluster Manager, Univa Grid Engine -- have incorporated support for containers. It is good to read some independent information about the role of containers in HPC.

Christian Kniep of Docker points out some issues with the interaction of HPC and Docker, and came up with a preliminary solution (a proxy for Docker Engine) to address the issues. HPC commonly makes use of specific hardware (e.g. GPUs, InfiniBand), and this is counter to Docker's hardware-agnostic approach. Also, HPC workflows may rely on shared resources (e.g. i/o to a shared filesystem).

The usefulness of awk in the present day

This anecdote illustrates exactly why scientists should invest some time in learning some of the standard tools of Unix/Linux.

https://news.ycombinator.com/item?id=17324122