docker build improvements (#5)

* docker build improvements

Improved docker build to use parametrized docker image base, allowing custom base images

Makefile target changes:
*  deb: performs a clean and runs `pg_buildext` to update `control` file before packaging
*  deb-docker instead of deb-in-docker, and can be parametrized - both the base image and extra packages
*  deb-latest runs in docker using `docker:latest` image
*  deb-sid runs in docker using `docker:sid` image

-----

TODO for another PR:

  When trying to build an image using Ubuntu (i guess just to see if it works), I get an error `Invalid cross-device link`. The probable cause is because docker maps two directories one inside the other, so perhaps there is a problem there. Not sure why it works fine with Debian base images.  Maybe @olksdr knows?

```
$ make deb-docker base=ubuntu:latest

   ....

dpkg-deb: building package 'postgresql-10-gzip' in '../postgresql-10-gzip_1.0.0_amd64.deb'.
dpkg-deb: building package 'postgresql-10-gzip-dbgsym' in 'debian/.debhelper/scratch-space/build-postgresql-10-gzip/postgresql-10-gzip-dbgsym_1.0.0_amd64.deb'.
	Renaming postgresql-10-gzip-dbgsym_1.0.0_amd64.deb to postgresql-10-gzip-dbgsym_1.0.0_amd64.ddeb
dh_builddeb: mv debian/.debhelper/scratch-space/build-postgresql-10-gzip/postgresql-10-gzip-dbgsym_1.0.0_amd64.deb ../postgresql-10-gzip-dbgsym_1.0.0_amd64.ddeb: Invalid cross-device link
dh_builddeb: Aborting due to earlier error
debian/rules:19: recipe for target 'binary-arch' failed
make[1]: *** [binary-arch] Error 2
make[1]: Leaving directory '/build/pgsql-gzip'
dpkg-buildpackage: error: fakeroot debian/rules binary-arch subprocess returned exit status 2
Makefile:46: recipe for target 'deb' failed
```

* Update README.md

Co-Authored-By: Oleksandr <[email protected]>

* Updated based on Olek's feedback

Remove the extras, but keep the base image (mostly useful for testing).
I will submit another PR in a few min to choose postgres version

* forgot minor cleanup

Co-authored-by: Oleksandr <[email protected]>

Author: nyurik

.dockerignore

@@ -0,0 +1,2 @@
+# Nothing should be passed to the docker build
+*

.gitignore

@@ -1,17 +1,21 @@
-*.o
-*.so
+*.a
+*.bc
 *.dll
 *.dylib
+*.o
 *.obj
-*.a
 *.pc
+*.so
+build*
+debian/.debhelper/
+debian/control
+debian/files
+debian/postgresql-*
+log/
+output_iso/
 regression.diffs
 regression.out
 results/
+target/
 tmp_check/
 tmp_check_iso/
-output_iso/
-log/
-target/
-build*
-*.bc

Dockerfile

@@ -1,12 +1,19 @@
-FROM debian:sid
+# This argument will usually be set by the Makefile. Override example:
+#  make deb-docker base=debian:latest
+ARG BASE_IMAGE=debian:sid
+FROM ${BASE_IMAGE}
 
-RUN apt-get update -qq && \
-  apt-get install -y \
-    build-essential \
-    fakeroot \
-    pkg-config \
-    devscripts \
-    zlib1g-dev \
-    postgresql-server-dev-all
+ARG BASE_IMAGE
+RUN set -eux  ;\
+    echo "Using BASE_IMAGE=$BASE_IMAGE"  ;\
+    DEBIAN_FRONTEND=noninteractive apt-get update -qq  ;\
+    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+        build-essential \
+        fakeroot \
+        pkg-config \
+        debhelper \
+        devscripts \
+        zlib1g-dev \
+        postgresql-server-dev-all
 
 WORKDIR /build/pgsql-gzip

Makefile

@@ -1,5 +1,5 @@
-# Make sure we do not run any code when using deb-in-docker target
-ifneq ($(MAKECMDGOALS),"deb-in-docker")
+# Make sure we do not run any code when using deb-* target
+ifeq (,$(findstring deb-,$(MAKECMDGOALS)))
 
 # Detect pkg-config on the path
 PKGCONFIG := $(shell type -p pkg-config || echo NONE)
@@ -32,7 +32,7 @@ LIBS += $(ZLIB_LIB)
 SHLIB_LINK := $(LIBS)
 
 ifdef DEBUG
-COPT			+= -O0 -g
+COPT += -O0 -g
 endif
 
 PGXS := $(shell $(PG_CONFIG) --pgxs)
@@ -42,14 +42,18 @@ endif
 
 
 .PHONY: deb
-deb:
+deb: clean
+	pg_buildext updatecontrol
 	dpkg-buildpackage -B
 
-.PHONY: deb-in-docker
-deb-in-docker: .image
-	mkdir -p "$$(pwd)/target"
-	docker run --rm -ti -v"$$(pwd)/target:/build" -v "$$(pwd):/build/pgsql-gzip" deb-builder make deb
-
-.PHONY: .image
-.image:
-	docker build -t deb-builder .
+# Name of the base Docker image to use. Uses debian:sid by default
+base ?= debian:sid
+
+.PHONY: deb-docker
+deb-docker:
+	@echo "*** Using base=$(base)"
+	docker build "--build-arg=BASE_IMAGE=$(base)" -t pgsql-gzip-$(base) .
+	# Create a temp dir that we will remove later. Otherwise docker will create a root-owned dir.
+	mkdir -p "$$(pwd)/target/pgsql-gzip"
+	docker run --rm -ti -u $$(id -u $${USER}):$$(id -g $${USER}) -v "$$(pwd)/target:/build" -v "$$(pwd):/build/pgsql-gzip" pgsql-gzip-$(base) make deb
+	rmdir "$$(pwd)/target/pgsql-gzip" || true

README.md

@@ -61,21 +61,41 @@ To convert a `bytea` back into an equivalent `text` you must use the `encode()`
 
 ### UNIX
 
-If you have PostgreSQL devel packages and zlib installed, you should have `pg_config` on your path, so you should be able to just run `make`, then `make install`, then in your database `CREATE EXTENSION gzip`.
+If you have PostgreSQL devel packages (`postgresql-server-dev-all`) and zlib (`zlib1g-dev`) installed, you should have `pg_config` on your path, so you should be able to just run `make`, then `make install`, then in your database `CREATE EXTENSION gzip`.
 
 If your `libz` is installed in a non-standard location, you may need to edit `ZLIB_PATH` in the `Makefile`.
 
-#### Debain/Ubuntu
+### Debain/Ubuntu
 
-To build the DEB package make use you have following dependencies installed as well:
+```bash
+sudo apt-get install build-essential zlib1g-dev postgresql-server-dev-all pkg-config
+make
+make install
+psql ... -c "CREATE EXTENSION gzip"
+```
+
+To build the DEB package you will also need `fakeroot` and `devscripts`. See [Dockerfile](./Dockerfile) for a full list.
 
 ```bash
-> apt-get install build-essential fakeroot devscripts
+sudo apt-get install build-essential zlib1g-dev postgresql-server-dev-all pkg-config fakeroot devscripts
+make
+make deb
+dpkg -i <path to package>.deb
 ```
 
-And you will be able to run the `make deb` and get the packege wich can be installed with `dpkg -i <path to package>.deb`
+And you will be able to run the `make deb` and get the packege wich can be installed with ``
+
+
+#### deb package build using Docker
 
+Makefile has targets for building the DEB package using Docker image with different base images. This approach only requires `make` and `docker` to be available on the host.
 
-#### In Docker
+```bash
+make deb-latest   # Uses debian:sid
+```
+
+To build an image using a different base, supply it with a parameter:
 
-Makefile has a special target for building the DEB packager directly in the docker by running `make deb-in-docker`. Where the `debian:sid` will be used to prepare the image with all the dependencies and the `make deb` target will be run inside it.
+```bash
+make deb-docker base=debian:latest
+```

debian/changelog

@@ -1,5 +1,7 @@
-pgsql-gzip (1.0.0) unstable; urgency=medium
+pgsql-gzip (1.0.1) unstable; urgency=medium
+  * DEB package cleanup, docker build improvements, readme updates [Yuri Astrakhan <[email protected]>]
+ -- Yuri Astrakhan <[email protected]>  Tue, 21 Apr 2020 22:21:00 +0000
 
+pgsql-gzip (1.0.0) unstable; urgency=medium
   * Initial release of the DEB package
-
  -- Oleksandr Kylymnychenko <[email protected]>  Fri, 20 Mar 2020 13:18:29 +0000