path: root/README-DEV
diff options
authorKaren Arutyunov <>2019-11-30 22:37:25 +0300
committerKaren Arutyunov <>2019-12-06 15:11:04 +0300
commitf1f39911e0d2d88c98eae96a3eb14a53c664206f (patch)
tree4cf4e3a84d895f59323d3b6ab4bfab38b3cab489 /README-DEV
parentfc9499b8a7b7a3e350bfabf2cd6ae0bc13f04bea (diff)
Upgrade to 12.1
Diffstat (limited to 'README-DEV')
1 files changed, 115 insertions, 97 deletions
diff --git a/README-DEV b/README-DEV
index acdefd9..c7ba5e1 100644
@@ -1,102 +1,120 @@
-This document describes how libpq was packaged for build2. In particular,
-this understanding will be useful when ugrading to a new upstream version.
-The original libpq library is packaged together with the PostgreSQL server and
-client utilities. Most of the libpq source files are located in the
-src/interfaces/libpq/ directory. Some .c files are symlinked (copied on
-Windows) from other src/ subdirectories during make. So run configure script
-in the package root, run make in src/interfaces/libpq/ and then copy source
-files and symlink targets into libpq/ directory of the build2 package. Note
-that to obtain the full set of source files that includes Windows-specific
-ones, you should perform these steps in the MSYS2/MinGW environment. Copy
-Windows-specific files and strlcpy.c to libpq/win32/ and libpq/non-bsd/
-directories respectively. Also copy src/port/pthread-win32.h to libpq/ (next
-to pthread-win32.c).
-Note that the library name in the .def file takes precedence over the one
-specified in the linker command line for both VC and MinGW GCC. So we comment
-it out in libpq/win32/libpqdll.def.
-Besides libpq-specific files some of the PostgreSQL common headers are also
-required. All of them except one (see below) are located in src/include/
-subdirectories. Copy them into the libpq/postgresql/ directory, preserving the
-original directory structure, with the following exceptions:
- * src/include/pg_config_os.h
- This is a symlink referencing the target-specific header in the
- src/include/port/ directory, created by the configure script. Copy these
- headers under postgresql/port/*/pg_config_os.h path names for the supported
- target classes. For example, copy src/include/port/linux.h to
- libpq/postgresql/port/linux/pg_config_os.h. During compilation the
- '-I.../libpq/postgresql/port/linux' option will be passed to the compiler,
- so the appropriate pg_config_os.h is picked up.
- * src/port/pg_config_paths.h
- This file is generated by src/port/Makfile and defines several directory
- path macros. Only SYSCONFDIR macro is used in libpq source files. Make the
- file empty and place it into libpq/postgresql directory. During compilation
- the macro will be defined with -DSYSCONFDIR preprocessor option.
- * src/include/,
- src/include/pg_config.h.win32
- pg_config_ext.h.win32
- Use as sources for the manual creation of libpq/postgresql/pg_config.h and
- libpq/postgresql/pg_config_ext.h, that are used for all target systems.
- Also add the source headers to the package appending the '.orig' extension
- to their names.
-By default the original package installs the library into the /usr/local/pgsql
-directory. There are also several 'unofficial API' headers installed into the
-/usr/local/pgsql/include/internal directory. If installing at some custom
-location that has no postgresql or pgsql words in its path, then it becomes
-<custom-location>/include/postgresql/internal. We currently do not install
-unofficial API headers.
-For the record, the PostgreSQL binary and development packages install
-libraries and headers into the following directories:
- /usr/lib/x86_64-linux-gnu
- /usr/include/postgresql
- /usr/include/postgresql/internal
- /usr/lib64
- /usr/include
- /usr/include/pgsql/internal
-When merge libpq build2 package with a new version of the original package
-make sure that all the preprocessor include directives reference the packaged
-header files, rather than PostgreSQL headers that are installed into the
-system. It's easy to miss some headers in the package if the PostgreSQL
-development package is installed on the host. To verify the correctness you
-can build the merged project, concatenate the produced .d files, sort the
-resulting file removing duplicates and edit the result, leaving only the
-system headers. Afterwards grep through the remained headers for the
-'PostgreSQL' pattern:
+This document describes an approach applied to packaging PostgreSQL for
+build2. In particular, this understanding will be useful when upgrading to a
+new upstream version.
-$ cat `find . -name '*.d'` | sort -u >headers
-$ emacs headers # Edit, leaving system headers only.
-$ fgrep PostgreSQL `cat headers`
+The upstream package contains the PostgreSQL server, libpq library, and client
+utilities. Currently, we only package libpq (see libpq/README-DEV for
+We add the upstream package as a git submodule and symlink the required files
+and subdirectories into the build2 package subdirectories. Then, when required,
+we "overlay" the upstream with our own headers, placing them into the library
+Note that symlinking upstream submodule subdirectories into a build2 package
+subdirectory results in creating intermediate build files (.d, .o, etc) inside
+upstream directory while building the package in source tree. That's why we
+need to make sure that packages do not share upstream source files via
+subdirectory symlinks, not to also share the related intermediate files. If
+several packages need to compile the same upstream source file, then only one
+of them can symlink it via the parent directory while others must symlink it
+directly. We also add the `ignore = untracked` configuration option into
+.gitmodules to make sure that git ignores the intermediate build files under
+upstream/ subdirectory.
+The upstream package can be configured to contain a specific feature set. We
+reproduce the union of features configured for the upstream source package in
+Debian and Fedora distributions. The configuration options defining these sets
+are specified in the Debian's rules and Fedora's RPM .spec files. Note,
+that at the time of this writing the latest packaged versions are different
+(12.1 on Debian and 11.6 on Fedora). That's probably ok since 12.1 doesn't
+introduce any new configuration options comparing to 11.6. These files can be
+obtained as follows:
+$ wget
+$ tar xf postgresql-12_12.1-1.debian.tar.xz debian/rules
+$ wget
+$ rpm2cpio libpq-11.6-1.fc32.src.rpm | cpio -civ '*.spec'
+As a side note, on Debian and Fedora the source, library, and headers are
+packaged as follows:
+ src libpq headers
+Debian/Ubuntu: postgresql-12 libpq5 libpq-dev
+Fedora/RHEL: libpq libpq libpq-devel
+Search for the Debian and Fedora packages at
+Here are the discovered configuration options.
+ --with-icu --with-tcl --with-perl --with-python --with-pam --with-openssl
+ --with-libxml --with-libxslt --enable-nls --enable-integer-datetimes
+ --with-gssapi --with-ldap --enable-thread-safety
+ --with-ldap --with-openssl --with-gssapi --enable-nls --without-readline
+The union of these feature sets translates into the following options, after
+suppressing the defaults:
-Also make sure that the macros set in libpq/postgresql/pg_config.h is still up
-to date. For that purpose obtain the macros that are used in the new source
-base, then obtain the macros (un)defined in the current
-libpq/postgresql/pg_config.h and compare the sets. That can be achieved
-running the following commands in the build2 project root directory:
+ --with-icu --with-tcl --with-perl --with-python --with-pam --with-openssl
+ --with-libxml --with-libxslt --enable-nls --with-gssapi --with-ldap
-$ for m in `cat libpq/postgresql/ libpq/postgresql/pg_config.h.win32.orig | sed -n 's/.*#\s*\(define\|undef\)\s\{1,\}\([_A-Z0-9]\{1,\}\)\(\s.*\)\{0,1\}$/\2/p' | sort -u`; do
- if grep -q -e "\b$m\b" `find . -name '*.h' -a ! -name 'pg_config.h' -o -name '*.c'`; then
- echo "$m"
- fi
- done >used-macros
+We, however, drop the external dependencies that are not packaged for build2
+and end up with the following options:
-$ cat libpq/postgresql/pg_config.h |
- sed -n 's/#\s*\(define\|undef\)\s\{1,\}\([_A-Z0-9]\{1,\}\)\(\s.*\)\{0,1\}$/\2/p' |
- sort -u >defined-macros
+ --with-openssl --without-readline --without-zlib
-$ diff defined-macros used-macros
+See the configuration options description at the "Installation Procedure" page
+Normally, when packaging a project, we need to replace some auto-generated
+headers with our own implementations and deduce compilation/linking options.
+For PostgreSQL we can rely on the,, and the
+manually maintained makefiles for that. In practice, however, that can be
+uneasy and error prone, so you may also need to refer to auto-generated
+header files or to see the actual compiler and linker command lines in the
+build log. If that's the case, you can configure/build the upstream package on
+the platform of interest running the following commands in the upstream
+project root directory.
+On POSIX and for MinGW GCC:
+$ mkdir build
+$ cd build
+$ ../configure --with-openssl --without-readline --without-zlib >build.log 2>&1
+$ cd src/interfaces/libpq
+$ make >>../../../build.log 2>&1
+See the "Installation from Source Code" page
+( for details.
+For MSVC:
+Add bison and flex to PATH, if building in the git repository. Install
+> cd src\tools\msvc
+Edit to enable OpenSSL:
+ openssl => 'c:\OpenSSL', # --with-openssl=<path>
+> build libpq >>build.log 2>&1
+See the "Building with Visual C++ or the Microsoft Windows SDK" page
+( for
+When the packaging is complete, build all the project packages in source tree
+and make sure that no PostgreSQL headers are included from the system, running
+the following commands from the project root:
+$ cat `find . -name '*.d'` | sort -u >headers
+$ emacs headers # Edit, leaving system headers only.
+$ fgrep PostgreSQL <headers