1 / 50

Advanced Composer - A Lot More Than Just ‘Install’ and ‘Update’

This workshop was presented at PHP South Africa 2017 (https://phpsouthafrica.com) Composer is one of the most important and useful tools in a PHP developer’s arsenal, however, a majority of its most powerful features go untouched by most. Join me for a deep dive through this versatile toolbox hiding in the root of your project. Highlights of this workshop include: * Leveraging Semantic Versioning and Private Packages to abstract and maintain integration of your microservice APIs * Composer Repositories as a means of high deployment availability when South Africa is on the far end of the Internet topology * Composer’s scripts and plugins as an important element of your deployment and testing pipeline. Buckle up for a slew of examples and recipes with a PHP DevOps flavour. The ideas are not perfect but they are definitely interesting.

bradm
Download Presentation

Advanced Composer - A Lot More Than Just ‘Install’ and ‘Update’

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Advanced composer A Lot More Than Just ‘Install’ and ‘Update’ PHP South Africa 2017 Brad Mostert

  2. Warning Buckle up for a slew of examples and recipes with a PHP DevOpsflavour. The ideas are not perfect but they are definitely interesting. /bsinkwa /mostertb

  3. Who am I Senior developer at Afrihost PHP Joburg Organizer Server Shepard Smart Giving the Design Patterns in PHP presentation on Friday Crazy

  4. Entrance test Have you done your pre-reading?

  5. Entrance test Install or Update: Newly downloaded software package (e.g. checking out some framework to start a new project) Answer: Install

  6. Entrance test Install or Update: Deploying code to server (committed lockfile) Answer: Install

  7. Entrance test Install or Update: Deploying code to server (no lockfilecommited) Answer: Install

  8. Entrance test Install or Update: Upgrade to the latest version of all your packages Answer: Update

  9. Entrance test Install or Update: You’ve manually edited your composer.json Answer: Update

  10. Environments Any Windows Users? Any Docker Users? Which Linux Distros?

  11. Composer Environment: Memory Limit Problem: Solution: composer_wrapper.sh php -d memory_limit=2G /path/to/global/composer.phar $@ .bashrc alias composer=‘/path/to/composer_wrapper.sh’ • Better Than: • Updating memory_limit in php.ini

  12. Composer Environment: Execution Time Problem: composer_wrapper.sh Solution: COMPOSER_PROCESS_TIMEOUT=600 php-d memory_limit=2G /path/to/global/composer.phar $@ Only affects the timeout of commands spawned by composer (e.g. git checkout) For individual projects: process-timeout node of composer.json

  13. Composer Environment: XDEBUG Problem: Solution: prompt $ composer self-update Fixed in version 1.3.0 (2016-12-24). Update your stuff you filthy developer Composer essentially makes a copy of you php.ini in /tmp with the line to load xdbug commented out, then restarts itself using the new config

  14. Composer Environment: oAuth Token Problem: OR Solution: prompt $ composer config -g github-oauth.github.com <oauthtoken> Create oAuth Token at: https://github.com/settings/tokens • Works for BitBucket and GitLab too • Also allows Auth for private repos • Good practice for CI, Build and Prod Servers Stored in ~/.composer/config.json

  15. Composer Environment: oAuth Token – Travis CI .travis.yml • Token not committed to code base • All builds will use the latest token on account • Not available to Pull Requests • Alternate: travis encrypt gem

  16. Production install flags • --prefer-dist • Super Github speed • Git export-ignore • --no-dev • Small deployment • Smaller attack surface • --optimize-autoloader • Power!

  17. Docker: GOAL Composer Install (and post-install scripts) take time to run. This is compounded when deploying to many servers in a HA environment Bake composer install into a production image so that deployment becomes: docker pull docker stop docker start Docker Rollout (circa 2017 colourized)

  18. Docker: Method One - Gameplan • Checkout just the composer.* files to docker build directory • Run docker build • Copy just composer.* files into build image • Run composer install (with --no-scripts) • Run composer clear-cache in same layer • Checkout project • composer dump-autoload • composer run-script

  19. Docker: Method One - Example build.sh … # Checkout composer.* files from Git repo at specific tag gitarchive --format=tar –remote=ssh://git@reposerver/repo.gittags/${1} -- composer.* | tar -xf– # Run docker build passing the git tag as an argument docker build -t ${DOCKER_IMAGE_NAME}:$1 . --build-argrolloutTag=${1} # Some error checking and ‘docker push’ go here … The first argument to the build script is the name of the tag in the Git repo that we are deploying. The new docker image will be tagged with this same tag

  20. Docker: Method One - Example Dockerfile … COPY composer.* ./ # Add your own prod flags for composer install RUN composer install --no-scripts && composer clear-cache ARG rolloutTag RUN git clone ssh://git@reposerver/repo.git/home/user/project/ && \gitfetch --all && \gitcheckout ${rolloutTag} RUN composer dump-autoload --optimize --no-dev RUN composer run-script post-install-cmd --no-dev# Optional …

  21. Docker: Method One - Review • Composer install only run if composer.* files change • No composer cache in image – smaller image • But no composer cache used during install – slow install

  22. Docker: Method Two Rube Goldberg Heath Robinson

  23. Docker: Method TWO - Gameplan • Run docker build to checkout codebase but don’t run composer install • Instantiate container from new image • Mount the build server’s composer cache as volume • Run composer install normally • Commit container as final image

  24. Docker: Method TWO - Example build.sh … docker build –t ${DOCKER_IMAGE_NAME}:${1}-checkout . --build-argrolloutTag=${1} docker run --name ${DOCKER_IMAGE_NAME}:${1}-composer--user prod_user \ -v ${PWD}/composer_cache:/home/prod_user/.composer \ ${DOCKER_IMAGE_NAME}:${1}-checkout composer install # Prod composer params docker commit \ --change="WORKDIR /home/user/project" \ --change='CMD ["apache2-foreground"]' \ --change="USER root” ${DOCKER_IMAGE_NAME}:${1}-composer \${DOCKER_IMAGE_NAME}:$1 # Cleanup ‘checkout’ image and ‘composer’ container # Push committed image

  25. Docker: Method TWO - Review • Composer install run with every build • Error handling in build script a little more complex (due to more steps) • Composer install fast due to using build server’s cache (saves bandwidth too) • Place for other run-time config

  26. Composer repositories ‘repositories’ node of composer.json allows you to specify package sources other than Packagist Note: Order is important. Composer will use the version of a package found in the first repository • Types: • VCS – git, svn, fossil or hg • – Github and BitBucket via API • PATH – directory on local machine • PEAR – yup • COMPOSER – Serve your own packages.json (like packagist) • PACKAGE – Treat some other project like a composer package

  27. Composer repositories: Path Allows you to install a package from another directory on the same machine Great for dev. Understands local VCS

  28. Composer repositories: Path Use with: & • Symlinks the path in to vendor directory: • Changes (including branch change) immediately available without composer update • Does behave as expected with ‘dev-branch’ version constrains . Rather use VCS repository if you need this • May occasionally require the autoloader to be re-dumped

  29. Composer repositories: Path Prevents: .gitconfig [alias] yolo = !"git commit -m \"$(echo $(curl -s http://whatthecommit.com/index.txt)) (whatthecommit.com)\"" Followed by a rebase…

  30. Composer repositories: Path Copy instead of Symlink: • Useful if you want to lock a a local package for a project • Branch version constraint needs to match what is currently checked out at install/update time • Getting updates to the package can be a pain (I generally delete the directory under vendor)

  31. local composer.json COMPOSER ENV Variable prompt $ COMPOSER=composer-local.json composer install • Large changes to composer.json for local environment? • Specify some other JSON file • Will generate .lock with the same name

  32. Composer repositories: VCS Intended for bug fixes to public packages As long as you keep the name node of the composer.json the same, the ‘order rule’ will mean that the version defined in your custom repository is used before the version from Packagist Type inferred from URL ( ‘ssh://’ , ‘.git’, etc.)

  33. Composer repositories: VCS Also supports private repos Leverages the auth credentials of the user (e.g. ssh keys, ~/.composer/config) Very Powerful URL can be a filesystem path if you prefer to dev with a gitflow/rebase approach rather than a ‘PATH’ repository

  34. Composer repositories: VCS Tagging a new version prompt $ git tag –a v1.2.3 –m “v1.2.3” $ git push $ git push --tags Sematic Versioning http://semver.org/

  35. Composer repositories: VCS Update a single package prompt # In the repos that use the package $ composer update afrihost/cloud-broker-client # Check composer.lock now has you new version $ gitdiff composer.lock # Commit composer.lock $ gitadd omposer.lock $ gitcommit -m 'Brad is awesum’ $ gitpush

  36. Composer repositories: VCS Maintaining Integration Between HTTP Microservices • Write HTTP Client Code as Private Package • Abstract API Behind PHP Functions/Classes • Functions return object (which has code completion) • Hide internal details like Transactional Consistency • Hide transport layer (e.g. Guzzle or Curl exceptions) • View your ‘interface’ for Semantic Versioning (breaking changes) in terms of PHP functions rather than API changes

  37. Composer repositories: VCS Private HTTP Client Versioning • Write HTTP Client Code as Private Package • Breaking changes to PHP Interface • Major version • Telling microservice maintainers that everything else can just be updated • Endpoint Additions • Point Release • Telling microservice maintainers “Hey there is a cool new thing” • Breaking HTTP API Changes • Minor Version • Works if Regular updates to all projects • Manage breaks over several versions

  38. Composer repositories: VCS Other Private HTTP Client Advantages • Ship non-sensitive config data in package • Default timeouts • CA Cert • Fits in well with Dependency Injection of frameworks • Good code reuse for integration code • Easier for developers to jump between codebases • Also works well for other cross-cutting concerns • Logging, Auth, Task Queuing, Templating

  39. Composer repositories: VCS Caveat Composer cant load repositories recursively – only repositories in the root package are inspected Problem if we want our private packages to make use of other private packages Solution: Repository Type COMPOSER

  40. Composer repositories: COMPOSER What Packagist provides packages.json Serve packages.json file which has details of available packages as well as its own schema elements (notify-batch, provider-includes and providers-url) @composer.json

  41. Composer repositories: COMPOSER You probably don’t want to hand-curate this • Toran Proxy (toranproxy.com) • Project of Jordi’s being replaced by Private Packagist • Private Packagist(packagist.com) • SaaS and Self Hosted options • Web Interface • Some revenue supports Composer and Packagisy.org development • Access Control • Web Hooks • Statis(github.com/composer/satis) • Free • CLI Based • Requires you to set up method of serving content and auth

  42. Composer repositories: STATIS statis.json Define all your private repositories as well as the range of package versions that you are interested in Run statis to generate a packages .json Still need to serve it yourself (https/ssh)

  43. Composer repositories: STATIS Update your projects to use COMPOSER repo composer.json

  44. Composer repositories: STATIS Mirror Content Improve speed Improve reliability (In South Africa, there is typically a lot of network between you and Github) Protection against original Git Repo disappearing ‘Offline Deployments’ (financial, remote)

  45. Composer repositories: STATIS Auth getcomposer.org/doc/articles/handling-private-packages-with-satis.md#security • SSH • Public/Private Key • HTTPS • Client Cert • HTTP Header • Authorized Bearer, API Token… • Basic Auth prompt Stored per project or globally (-g) in ~/.composer/auth.json

  46. GIT ARCHIVE Exclude stuff from Dist versions .gitattributes /Tests export-ignore /.gitattributes export-ignore /.travis.yml export-ignore /phpunit.xml export-ignore Github uses ‘git archive’ to generate the release artefacts

  47. Prefer-lowest Prefer-Lowest in CI/Test Chain .travis.yml

  48. Scripts Run PHP Callback, PHP Script or CLI Commands at Event during composer lifecycle

  49. Scripts: ideas COMPOSER_DEV_MODE env variable during install and update with value for --no-dev • Framework steps • cache clear, asset setup • SensioDistributionBundle • Build chains outside of PHP (npm,gulp) • Deployment notification • Continuous Deployment, Slack Notification • Track Private Package version usage on Production Hosts (extension to ‘notify-batch’) • Run test suite after install or update

  50. Questions? PHP South Africa 2017 Brad Mostert /bsinkwa /mostertb

More Related