Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: edolstra/nix-serve
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: jskrzypek/nix-serve-nothing
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Can’t automatically merge. Don’t worry, you can still create the pull request.
  • 9 commits
  • 5 files changed
  • 2 contributors

Commits on Aug 28, 2020

  1. Rewrite the server & nix derivations

    The server should always return 404s except for the nix store info. The
    nix files should build the package & the service.
    jskrzypek committed Aug 28, 2020
    Copy the full SHA
    2cf0653 View commit details

Commits on Aug 29, 2020

  1. fix: Derivation should not refer to GH

    That was kind of dumb. The derivation should take its source from the
    current dir.
    jskrzypek committed Aug 29, 2020
    Copy the full SHA
    c3c1de8 View commit details

Commits on Aug 30, 2020

  1. Add support for proxying to remote cache

    If we want to be able to use a remote cache sometimes, we should proxy
    binary requests to it based on criteria that we pass on the command
    line.
    jskrzypek committed Aug 30, 2020
    Copy the full SHA
    6ed71cb View commit details

Commits on Aug 31, 2020

  1. Fix issues with server

    The whole our system is weird, but this seems to work now and to
    correctly proxy requests.
    jskrzypek committed Aug 31, 2020
    Copy the full SHA
    ee6392f View commit details
  2. Fix service module & use DynamicUser

    This fixes the way the service module invokes the derivation and
    switches to DynamicUser to avoid the extraneous user config.
    
    Also bump the version.
    jskrzypek committed Aug 31, 2020
    Copy the full SHA
    c682689 View commit details

Commits on Sep 1, 2020

  1. Add cache behavior to command result

    Instead of determining if to use the proxy once for the whole lifetime
    of the server, and then requiring a restart when it changes, we can
    accept an extra proxy option that will re-run the check of the command
    after a set interval. The default is 300s or 5 minutes.
    jskrzypek committed Sep 1, 2020
    Copy the full SHA
    0cfe7f8 View commit details
  2. fix: Correctly handle int & float proxy args

    The cachettl arg is an int, but we were ignoring it in the mkProxy
    function.
    jskrzypek committed Sep 1, 2020
    Copy the full SHA
    068b63c View commit details

Commits on Sep 2, 2020

  1. Make server work with env vars

    The command line args aren't working with starman. I hate perl. Also wtf
    scalar and localtime and select. FML.
    jskrzypek committed Sep 2, 2020
    Copy the full SHA
    a81445f View commit details

Commits on Sep 5, 2020

  1. Add nixos install example

    Add a full example of installation in nixos, including importing the nix
    module and specifying a conditional proxy.
    Griffin Smith authored and glittershark committed Sep 5, 2020
    Copy the full SHA
    6084849 View commit details
Showing with 303 additions and 86 deletions.
  1. +55 −0 README.md
  2. +35 −16 default.nix
  3. +96 −0 nix-serve-nothing.psgi
  4. +0 −70 nix-serve.psgi
  5. +117 −0 service.nix
55 changes: 55 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# nix-serve-nothing

This is a hacky little fork of Eelco Dolstra's [nix-serve](https://github.com/edolstra/nix-serve) that lets us serve
responses that nix can understand and avoid a roundtrip call to a binary cache.

If you have a binary cache sitting behind a VPN/Firewall running nix-build
requires you to be on the VPN, or else nix hangs for a while until the request
times out and then fails. By providing a server that always claims to not have
the requested binaries we can force nix to move on to asking the next cache, or
even just building the binaries itself.

The server also supports passing a command that it can use to check if requests
should be proxied to a gonfigurable target server. If the check exits with 0 or
returns a specified value to STDOUT then the server will proxy to the binary
cache server specified at startup. Otherwise it will respond with 404s. This
allows us to impersonate and return the null responses for our private VPNed
binary cache without needing to impersonate its certificates.

## Installation

### NixOS

``` nix
imports = [
"${(fetchTarball "https://github.com/jskrzypek/nix-serve-nothing/archive/0cfe7f8a4b3aed152be590885deefadb1324c732.tar.gz")}/service.nix"
];
nix.binaryCaches = [ "http://localhost:13823" ];
services.nix-serve-nothing = {
enable = true;
bindAddress = "localhost";
port = 13823;
proxy = {
enable = true;
target = "https://your.nix.cache.here";
# Command to run to determine whether or not to proxy
command = "systemctl is-active openvpn-your-vpn.service";
# If the command outputs this string, proxy
when = "active";
};
};
```

## Options

The service specifies configuring options. Otherwise the following parameters
can be passed on the command line to the PSGI server:

* **listen** - The address on which to listen
* **proxy** - (*optional*) A hash that configures the proxy
* **target** - The remote binary cache to proxy requests to
* **command** - The shell command to determine if requests should be proxied
* **when** - (*optional*) A value to match the output of command against
* **cachettl** - (*optional - default 300*) The number of seconds to wait before checking the command again.
51 changes: 35 additions & 16 deletions default.nix
Original file line number Diff line number Diff line change
@@ -1,24 +1,43 @@
with import <nixpkgs> {};
{ pkgs, debug ? false, ... }:

let nix = nixUnstable; in
with pkgs;
with lib;
with stdenv.lib;

stdenv.mkDerivation {
name = "nix-serve-2";
pname = "nix-serve-nothing";
version = "0.5";

buildInputs = [ perl nix perlPackages.Plack perlPackages.Starman perlPackages.DBDSQLite ];
src = ./.;

unpackPhase = "true";
buildInputs = [ bzip2 perl nix nix.perl-bindings ]
++ (with perlPackages;
[ GetoptLong LWPProtocolHttps Plack PlackAppProxy StringShellQuote Starman ]);

installPhase =
''
mkdir -p $out/libexec/nix-serve
cp ${./nix-serve.psgi} $out/libexec/nix-serve/nix-serve.psgi
dontBuild = true;

mkdir -p $out/bin
cat > $out/bin/nix-serve <<EOF
#! ${stdenv.shell}
PERL5LIB=$PERL5LIB exec ${perlPackages.Starman}/bin/starman $out/libexec/nix-serve/nix-serve.psgi "\$@"
EOF
chmod +x $out/bin/nix-serve
'';
installPhase = ''
mkdir -p $out/libexec/nix-serve-nothing
cp nix-serve-nothing.psgi $out/libexec/nix-serve-nothing/nix-serve-nothing.psgi
mkdir -p $out/bin
cat > $out/bin/nix-serve-nothing <<EOF
#! ${stdenv.shell}
NIX_SERVE_NOTHING_DEBUG=${toString debug} PATH=${makeBinPath [ bzip2 nix ]}:\$PATH PERL5LIB=$PERL5LIB exec ${perlPackages.Starman}/bin/starman $out/libexec/nix-serve-nothing/nix-serve-nothing.psgi "\$@"
EOF
chmod +x $out/bin/nix-serve-nothing
'';

meta = {
homepage = "https://github.com/jskrzypek/nix-serve-nothing";
description = "A utility for sharing absolutely nothing as a binary cache";
maintainers = [{
email = "jskrzypek@gmail.com";
github = "jskrzypek";
githubId = 1513265;
name = "Joshua Skrzypek";
}];
license = licenses.lgpl21;
platforms = nix.meta.platforms;
};
}
96 changes: 96 additions & 0 deletions nix-serve-nothing.psgi
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
use POSIX;
use Nix::Config;
use String::ShellQuote;
use Getopt::Long;
use Plack::App::Proxy;
use strict;

use constant DEBUG => $ENV{NIX_SERVE_NOTHING_DEBUG} || 0;
select(STDOUT);

my %proxyOpts = (
target => $ENV{PROXY_TARGET} || "",
command => $ENV{PROXY_COMMAND} || "",
when => $ENV{PROXY_WHEN} || "",
cachettl => $ENV{PROXY_CACHETTL} || 300,
);
$proxyOpts{'cachettl'} = int($proxyOpts{'cachettl'});
my $useProxy = "";
my $proxy = Plack::App::Proxy->new(
remote => $proxyOpts{'target'},
backend => 'LWP',
)->to_app;
my $checkedAt = 0;
GetOptions ('proxy-opts|p:s%{2,4}' => \%proxyOpts);

print scalar localtime;
print ": Booting server with proxy-options\n";
print " * target : $proxyOpts{'target'}\n";
print " * command : $proxyOpts{'command'}\n";
print " * when : $proxyOpts{'when'}\n";
print " * cachettl : $proxyOpts{'cachettl'}\n";

# we don't just check at startup so we can be lazy and wait for the state when
# first queried
sub checkUseProxy {
my $now = time;
my $nextcheck = int($checkedAt) + int($proxyOpts{'cachettl'});

DEBUG && print(scalar(localtime), ": Last checked command at ", scalar(localtime($checkedAt)), "\n");
DEBUG && print(scalar(localtime), ": Check scheduled for ", scalar(localtime($nextcheck)), "\n");

if ($now > $nextcheck) {
$checkedAt = $now;
DEBUG && print(scalar(localtime), ": Checking command now...");
my $cmd_ret_val = readpipe(shell_quote split(" ", $proxyOpts{'command'}));
my $cmd_exit_code = $?;

DEBUG && print scalar(localtime), ": Command `";
DEBUG && print shell_quote split(" ", $proxyOpts{'command'});
DEBUG && print "` exited with $cmd_exit_code and returned '$cmd_ret_val'.\n";

$useProxy = 0;

if ($proxyOpts{'when'} && $cmd_ret_val == $proxyOpts{'when'}) {
$useProxy = 1;
}

elsif ($cmd_exit_code == 0) {
$useProxy = 1;
}

if ($useProxy == 1 && length $proxyOpts{'target'} > 0) {
print scalar localtime;
print ": Nix-serve-nothing is now proxying to ";
print $proxyOpts{'target'};
print ".\n";
}
else {
print scalar localtime;
print ": Nix-serve-nothing is now resoponding without proxy.\n";
}
}


return $useProxy;
}

my $app = sub {
my $env = shift;

my $path = $env->{PATH_INFO};


if (checkUseProxy() == 1) {
print "$env->{REQUEST_METHOD} <$proxyOpts{'target'}> $path\n";
return $proxy->($env);
}

print "$env->{REQUEST_METHOD} $path\n";

if ($path eq "/nix-cache-info") {
return [200, ['Content-Type' => 'text/plain'], ["StoreDir: $Nix::Config::storeDir\nWantMassQuery: 1\nPriority: 30\n"]];
}

return [404, ['Content-Type' => 'text/plain'], ["File not found.\n"]];
}
70 changes: 0 additions & 70 deletions nix-serve.psgi

This file was deleted.

Loading