PyDev, DevOps
0,0
рейтинг
4 ноября 2013 в 19:45

Разработка → Универсальный способ создания пакетов, для различных дистрибутивов GNU Linux tutorial

Ruby*
Различные дистрибутивы предоставляют свои утилиты, для сборки и установки программ.
Например в Debian/GNU Linux — это debuild и dpkg. В Red Hat Linux — rpmbuild и rpm.
Не редко нам приходится собирать пакеты самостоятельно.
Устанавливать программы через make, в обход системы управления пакетами в дистрибутивах — является дурным тоном.
В случаях, когда мы имеем исходный код, и автор программы позаботился о создании Makefile, можно использовать checkinstall.
Но бывает и так, что автор java приложения, не выкладывает ничего кроме jar файла. Или мы хотим на коленке создать какой-то пакет, и положить в репозиторий, что бы в дальнейшем устанавливать/обновлять его на множестве серверов с различными дистрибутивами. В этом нам поможет утилита под названием fpm.
В чем же ее достоинство? Она из коробки позволяет нам собирать пакеты под различные дистрибутивы и даже операционные системы.

Сейчас поддерживаются:
  • deb
  • rpm
  • solaris
  • tar
  • directories
  • Mac OS X .pkg files (osxpkg)


Установка необходимых пакетов для работы с fpm в Debian GNU/Linux:
# apt-get install ruby rubygems rpm dpkg-dev
# gem install fpm

Для примера соберем пакет logstash, для Debian/GNU Linux и Red Hat Linux.
Создадим файл сборки, в котором опишем зависимости, файлы конфигурации и мета-информацию о пакете:
$ mkdir logstash && cd logstash && touch build.sh && chmod +x build.sh && vim build.sh

Вставляем:
#!/bin/bash

JAR_URL="https://download.elasticsearch.org/logstash/logstash/logstash-1.2.2-flatjar.jar"
JAR_FILE="usr/share/logstash/logstash.jar"

DESCRIPTION="Logstash Open Source Log Management" 
EMAIL="admin@logstash.net"
URL='http://www.logstash.net/'
VERSION="1.2.2"
NAME="logstash"

# создаем структуру директорий
if [ ! -d "usr" ]; then
	mkdir -p {etc,usr/share/logstash,var/logstash}
fi

# скачиваем и кладем в необходимый каталог logstash
if [ ! -f "$JAR_FILE" ]; then
	wget "$JAR_URL" -O "$JAR_FILE"
fi

cd ..

function build() {
    fpm -n $NAME -v $VERSION -a all -C logstash -m "<$EMAIL>" \
        --pre-install logstash/preinstall \
        --description "$DESCRIPTION" \
        --url "$URL" -t "$1" -d "$2" \
        --config-files etc/logstash/syslog.conf \
        -s dir etc usr var
}

# задаем тип пакета и зависимости
build "deb" "default-jre"
build "rpm" "java-1.6.0-openjdk-devel"


Создадим файл конфигурации logstash:
$ mkdir -p etc/logstash && vim etc/logstash/syslog.conf

вставляем:
etc/logstash/syslog.conf
input {
  file {
    type => "syslog_file"
    exclude => [ "logstash.log*" ]
    path => [ "/var/log/messages", "/var/log/syslog", "/var/log/*.log" ]
  }
}

filter {
  grok {
    type => "syslog_relay"
    pattern => [ "^<[1-9]\d{0,2}>%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_pri"
    add_field => [ "syslog_raw_message", "%{@message}" ]
  }
  syslog_pri {
    type => "syslog_relay"
    tags => [ "got_syslog_pri" ]
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_pri" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_pri"
  }

  grok {
    type => "syslog_relay"
    pattern => [ "^%{SYSLOGTIMESTAMP:syslog_timestamp}%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_timestamp"
    add_field => [ "received_at", "%{@timestamp}" ]
  }
  date {
    type => "syslog_relay"
    tags => [ "got_syslog_timestamp" ]
    # season to taste for your own syslog format(s)
    match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601" ]
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_timestamp" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_timestamp"
  }

  grok {
    type => "syslog_relay"
    pattern => [ "^%{SYSLOGHOST:syslog_hostname}%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_host"
    add_field => [ "logstash_source", "%{@source_host}" ]
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_host" ]
    replace => [ "@source_host", "%{syslog_hostname}" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_host"
  }

  grok {
    type => "syslog_relay"
    pattern => [ "^%{SYSLOGPROG:syslog_prog}:%{SPACE}%{GREEDYDATA:message_remainder}" ]
    add_tag => "got_syslog_prog"
  }
  mutate {
    type => "syslog_relay"
    tags => [ "got_syslog_prog" ]
    replace => [ "@message", "%{message_remainder}" ]
    remove => [ "message_remainder" ]
    remove_tag => "got_syslog_prog"
  }

  dns {
    type => 'syslog_relay'
    reverse => [ "@source_host", "@source_host" ]
    action => "replace"
  }

  mutate {
    type => "syslog_relay"
    tags => [ "syslog" ]
    remove => [ "syslog_hostname", "syslog_timestamp" ]
  }
}



Далее создадим скрипт, который должен выполнится перед установкой пакета:
$ cat > preinstall  << END
#!/bin/bash
useradd -g adm -r -m -d /usr/share/logstash -s /bin/false logstash || exit 0
END

Теперь достаточно запустить сборку пакетов:
# ./build.sh

На выходе мы получим rpm и deb пакеты, которые без проблем устанавливаются как в Debian, так и в Red Hat.

Устанавливаем пакет на текущей машине и проверяем:
# dpkg -i ../logstash*.deb && apt-get install -f
# java -jar /usr/share/logstash/logstash.jar agent -f /etc/logstash --log /var/log/logstash.log


Заключение


Буквально несколькими командами можно собрать пакет под различные дистрибутивы и ОС за пару минут.
Спасибо автору за столь прекрасную утилиту. Надеюсь, что кто-то ее найдет такой же полезной как и я.
Руслан @gotlium
карма
15,7
рейтинг 0,0
PyDev, DevOps
Реклама помогает поддерживать и развивать наши сервисы

Подробнее
Спецпроект

Самое читаемое Разработка

Комментарии (6)

  • +2
    Интересно, а с более сложными исходниками оно справится? PHP или nginx?.. По мне, сравнивая приведенный выше скрипт сборки и спек файл — последний более понятнее и гибче в настройке.

    скачиваем и ложим в необходимый каталог logstash

    Кладем.
    • 0
      Данная штука позволяет быстро создать обычный пакет. Для nginx или php — подходит лучше checkintsall.
      • 0
        А что для вас обычный пакет?.. Просто набор файлов и команда создания пользователя? Спекфайлы тоже позволяют это делать :)
        А лог эта штука дает?
        • 0
          Приблизительно так. Как по мне, так control файлы гибче. Лог выдает на stdout.
          Возможности:
          Usage
          Usage:
          fpm [OPTIONS] [ARGS]…

          Parameters:
          [ARGS]… Inputs to the source package type. For the 'dir' type, this is the files and directories you want to include in the package. For others, like 'gem', it specifies the packages to download and use as the gem input

          Options:
          -t OUTPUT_TYPE the type of package you want to create (deb, rpm, solaris, etc)
          -s INPUT_TYPE the package type to use as input (gem, rpm, python, etc)
          -C CHDIR Change directory to here before searching for files
          --prefix PREFIX A path to prefix files with when building the target package. This may be necessary for all input packages. For example, the 'gem' type willprefix with your gem directory automatically.
          -p, --package OUTPUT The package file path to output.
          -f, --force Force output even if it will overwrite an existing file (default: false)
          -n, --name NAME The name to give to the package
          --verbose Enable verbose output
          --debug Enable debug output
          -v, --version VERSION The version to give to the package (default: 1.0)
          --iteration ITERATION The iteration to give to the package. RPM calls this the 'release'. FreeBSD calls it 'PORTREVISION'. Debian calls this 'debian_revision'
          --epoch EPOCH The epoch value for this package. RPM and Debian calls this 'epoch'. FreeBSD calls this 'PORTEPOCH'
          --license LICENSE (optional) license name for this package
          --vendor VENDOR (optional) vendor name for this package
          --category CATEGORY (optional) category this package belongs to (default: «none»)
          -d, --depends DEPENDENCY A dependency. This flag can be specified multiple times. Value is usually in the form of: -d 'name' or -d 'name > version' (default: [])
          --no-depends Do not list any dependencies in this package (default: false)
          --no-auto-depends Do not list any dependencies in this package automatically (default: false)
          --provides PROVIDES What this package provides (usually a name). This flag can be specified multiple times.
          --conflicts CONFLICTS Other packages/versions this package conflicts with. This flag can specified multiple times.
          --replaces REPLACES Other packages/versions this package replaces. This flag can be specified multiple times.
          --config-files CONFIG_FILES Mark a file in the package as being a config file. This uses 'conffiles' in debs and %config in rpm. If you have multiple files to mark as configuration files, specify this flag multiple times.
          --directories DIRECTORIES Mark a directory as being owned by the package
          -a, --architecture ARCHITECTURE The architecture name. Usually matches 'uname -m'. For automatic values, you can use '-a all' or '-a native'. These two strings will be translated into the correct value for your platform and target package type.
          -m, --maintainer MAINTAINER The maintainer of this package. (default: "<jls@sadness>")
          -S, --package-name-suffix PACKAGE_NAME_SUFFIX a name suffix to append to package and dependencies.
          -e, --edit Edit the package spec before building. (default: false)
          -x, --exclude EXCLUDE_PATTERN Exclude paths matching pattern (shell wildcard globs valid here). If you have multiple file patterns to exclude, specify this flag multiple times.
          --description DESCRIPTION Add a description for this package. You can include '
          ' sequences to indicate newline breaks. (default: «no description»)
          --url URI Add a url for this package. (default: «example.com/no-uri-given»)
          --inputs INPUTS_PATH The path to a file containing a newline-separated list of files and dirs to use as input.
          --post-install FILE (DEPRECATED, use --after-install) a script to be run after package installation
          --pre-install FILE (DEPRECATED, use --before-install) a script to be run before package installation
          --post-uninstall FILE (DEPRECATED, use --after-remove) a script to be run after package removal
          --pre-uninstall FILE (DEPRECATED, use --before-remove) a script to be run before package removal
          --after-install FILE a script to be run after package installation
          --before-install FILE a script to be run before package installation
          --after-remove FILE a script to be run after package removal
          --before-remove FILE a script to be run before package removal
          --template-scripts Allow scripts to be templated. This lets you use ERB to template your packaging scripts (for --after-install, etc). For example, you can do things like <%= name %> to get the package name. For more information, see the fpm wiki: github.com/jordansissel/fpm/wiki/Script-Templates
          --template-value KEY=VALUE Make 'key' available in script templates, so <%= key %> given will be the provided value. Implies --template-scripts
          --workdir WORKDIR The directory you want fpm to do its work in, where 'work' is any filecopying, downloading, etc. Roughly any scratch space fpm needs to buildyour package. (default: "/tmp")
          --gem-bin-path DIRECTORY (gem only) The directory to install gem executables (default: "/home/jls/.rvm/gems/ruby-2.0.0-p0/bin")
          --gem-package-prefix NAMEPREFIX (gem only) (DEPRECATED, use --package-name-prefix) Name to prefix the package name with.
          --gem-package-name-prefix PREFIX (gem only) Name to prefix the package name with. (default: «rubygem»)
          --gem-gem PATH_TO_GEM (gem only) The path to the 'gem' tool (defaults to 'gem' and searches your $PATH) (default: «gem»)
          --[no-]gem-fix-name (gem only) Should the target package name be prefixed? (default: true)
          --[no-]gem-fix-dependencies (gem only) Should the package dependencies be prefixed? (default: true)
          --[no-]gem-env-shebang (gem only) Should the target package have the shebang rewritten to use env? (default: true)
          --[no-]gem-prerelease (gem only) Allow prerelease versions of a gem (default: false)
          --[no-]deb-ignore-iteration-in-dependencies (deb only) For '=' (equal) dependencies, allow iterations on the specified version. Default is to be specific. This option allows the same version of a package but any iteration is permitted
          --deb-pre-depends DEPENDENCY (deb only) Add DEPENDENCY as a Pre-Depends
          --deb-compression COMPRESSION (deb only) The compression type to use, must be one of gz, bzip2, xz. (default: «gzip»)
          --deb-custom-control FILEPATH (deb only) Custom version of the Debian control file.
          --deb-config SCRIPTPATH (deb only) Add SCRIPTPATH as debconf config file.
          --deb-templates FILEPATH (deb only) Add FILEPATH as debconf templates file.
          --deb-installed-size KILOBYTES (deb only) The installed size, in kilobytes. If omitted, this will be calculated automatically
          --deb-priority PRIORITY (deb only) The debian package 'priority' value. (default: «extra»)
          --deb-user USER (deb only) The owner of files in this package
          --deb-group GROUP (deb only) The group owner of files in this package
          --deb-changelog FILEPATH (deb only) Add FILEPATH as debian changelog
          --deb-recommends PACKAGE (deb only) Add PACKAGE to Recommends
          --deb-suggests PACKAGE (deb only) Add PACKAGE to Suggests
          --deb-field 'FIELD: VALUE' (deb only) Add custom field to the control file
          --deb-shlibs SHLIBS (deb only) Include control/shlibs content. This flag expects a string that is used as the contents of the shlibs file. See the following url for a description of this file and its format: www.debian.org/doc/debian-policy/ch-sharedlibs.html#s-shlibs
          --[no-]rpm-use-file-permissions (rpm only) Use existing file permissions when defining ownership and modes
          --rpm-user USER (rpm only) Set the user to USER in the %files section. (default: «root»)
          --rpm-group GROUP (rpm only) Set the group to GROUP in the %files section. (default: «root»)
          --rpm-rpmbuild-define DEFINITION (rpm only) Pass a --define argument to rpmbuild.
          --rpm-digest md5|sha1|sha256|sha384|sha512 (rpm only) Select a digest algorithm. md5 works on the most platforms. (default: «md5»)
          --rpm-compression none|xz|gzip|bzip2 (rpm only) Select a compression method. gzip works on the most platforms. (default: «gzip»)
          --rpm-os OS (rpm only) The operating system to target this rpm for. You want to set this to 'linux' if you are using fpm on OS X, for example
          --rpm-changelog FILEPATH (rpm only) Add changelog from FILEPATH contents
          --[no-]rpm-sign (rpm only) Pass --sign to rpmbuild
          --python-bin PYTHON_EXECUTABLE (python only) The path to the python executable you wish to run. (default: «python»)
          --python-easyinstall EASYINSTALL_EXECUTABLE (python only) The path to the easy_install executable tool (default: «easy_install»)
          --python-pip PIP_EXECUTABLE (python only) The path to the pip executable tool. If not specified, easy_install is used instead (default: nil)
          --python-pypi PYPI_URL (python only) PyPi Server uri for retrieving packages. (default: «pypi.python.org/simple»)
          --python-package-prefix NAMEPREFIX (python only) (DEPRECATED, use --package-name-prefix) Name to prefix the package name with.
          --python-package-name-prefix PREFIX (python only) Name to prefix the package name with. (default: «python»)
          --[no-]python-fix-name (python only) Should the target package name be prefixed? (default: true)
          --[no-]python-fix-dependencies (python only) Should the package dependencies be prefixed? (default: true)
          --[no-]python-downcase-name (python only) Should the target package name be in lowercase? (default: true)
          --[no-]python-downcase-dependencies (python only) Should the package dependencies be in lowercase? (default: true)
          --python-install-bin BIN_PATH (python only) The path to where python scripts should be installed to. (default: "/usr/bin")
          --python-install-lib LIB_PATH (python only) The path to where python libs should be installed to (default depends on your python installation). Want to what your target platform is using? Run this: python -c 'from distutils.sysconfig import get_python_lib; print get_python_lib()'
          --python-install-data DATA_PATH (python only) The path to where data should be.installed to. This is equivalent to 'python setup.py --install-data DATA_PATH
          --[no-]python-dependencies (python only) Include requirements defined in setup.py as dependencies. (default: true)
          --[no-]python-obey-requirements-txt (python only) Use a requirements.txt filein the top-level directory of the python package for dependency detection. (default: false)
          --osxpkg-identifier-prefix IDENTIFIER_PREFIX (osxpkg only) Reverse domain prefix prepended to package identifier, ie. 'org.great.my'. If this is omitted, the identifer will be the package name.
          --[no-]osxpkg-payload-free (osxpkg only) Define no payload, assumes use of script options. (default: false)
          --osxpkg-ownership OWNERSHIP (osxpkg only) --ownership option passed to pkgbuild. Defaults to 'recommended'. See pkgbuild(1). (default: «recommended»)
          --osxpkg-postinstall-action POSTINSTALL_ACTION (osxpkg only) Post-install action provided in package metadata. Optionally one of 'logout', 'restart', 'shutdown'.
          --osxpkg-dont-obsolete DONT_OBSOLETE_PATH (osxpkg only) A file path for which to 'dont-obsolete' in the built PackageInfo. Can be specified multiple times.
          --solaris-user USER (solaris only) Set the user to USER in the prototype files. (default: «root»)
          --solaris-group GROUP (solaris only) Set the group to GROUP in the prototype file. (default: «root»)
          --npm-bin NPM_EXECUTABLE (npm only) The path to the npm executable you wish to run. (default: «npm»)
          --npm-package-name-prefix PREFIX (npm only) Name to prefix the package name with. (default: «node»)
          --pear-package-name-prefix PREFIX (pear only) Name prefix for pear package (default: «php-pear»)
          --pear-channel CHANNEL_URL (pear only) The pear channel url to use instead of the default.
          --[no-]pear-channel-update (pear only) call 'pear channel-update' prior to installation
          --pear-bin-dir BIN_DIR (pear only) Directory to put binaries in
          --pear-php-bin PHP_BIN (pear only) Specify php executable path if differs from the os used for packaging
          --pear-php-dir PHP_DIR (pear only) Specify php dir relative to prefix if differs from pear default (pear/php)
          -h, --help print help
          • 0
            Ладно, попробуем на выходных поиграться :) Спасибо за статью
            • 0
              Пожалуйста. Штука в целом интересная.

Только зарегистрированные пользователи могут оставлять комментарии. Войдите, пожалуйста.