<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Herberto Graça on Medium]]></title>
        <description><![CDATA[Stories by Herberto Graça on Medium]]></description>
        <link>https://medium.com/@herberto.graca?source=rss-6cdf53b3244------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*oyiRsKcX2EiTmJYRXGjNAw.jpeg</url>
            <title>Stories by Herberto Graça on Medium</title>
            <link>https://medium.com/@herberto.graca?source=rss-6cdf53b3244------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:46:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@herberto.graca/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Resource-Method-Representation]]></title>
            <link>https://medium.com/the-software-architecture-chronicles/resource-method-representation-61c7dea1217f?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/61c7dea1217f</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[rmr]]></category>
            <category><![CDATA[mvc]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Fri, 31 Aug 2018 07:30:50 GMT</pubDate>
            <atom:updated>2018-08-31T17:06:28.171Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a><em>, a </em><a href="https://herbertograca.com/category/development/series/software-architecture/"><em>series of posts about Software Architecture</em></a><em>. In them, I write about what I’ve learned about Software Architecture, how I think of it, and how I use that knowledge. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>MVC was created in 1979, in a context of desktop applications with CLI user interfaces and it implied that the UI would be changed automatically if there were changes in the database, caused by some factor external to the user. The same pattern was perfectly usable later on desktop applications with a GUI.</p><p>However, its usage in web applications has always been an adaptation because most web applications don’t change the UI as a consequence of changes that happen in the server side, there is always a call from the UI asking the server side for an update of the screen.</p><p>I have talked about <a href="https://herbertograca.com/2017/08/17/mvc-and-its-variants/">variants of the MVC pattern before</a>, this post is about another variant: <strong><em>Resource-Method-Representation</em></strong>.</p><p>I feel the need to talk about it, not because I find it a key pattern in my practice but because of the misconception that it is the same as the ADR pattern, of which I will write about soon.</p><h3>2008 — Resource-Method-Representation</h3><p>The <strong>RMR</strong> pattern was created by Paul James in 2008 and it adapts the MVC pattern to the context of REST APIs.</p><h3>Resource</h3><p>The idea is that the Entities are modelled as REST resources (the first R in the pattern name), with its only public methods mapping to an HTTP method:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/67f74438ae4aa2dc4c5fdda8194e87fd/href">https://medium.com/media/67f74438ae4aa2dc4c5fdda8194e87fd/href</a></iframe><h3>Method</h3><p>When a request is made to the API, it is routed to one of these business objects, the Resources, and the method that is called on this resource corresponds to the HTTP method of the request. This method on this business object is then responsible for returning a complete <em>http</em> response including its status code and headers.</p><h3>Representation</h3><p>The representation is the Resource representation in a format chosen by the API or requested by the client, ie. JSON, XML, etc.. This representation is the content of the response created by the Method and sent back to the client if/when there is any content to be sent back.</p><h3>My take on this pattern</h3><p>The MVC pattern is a presentation pattern, it is a way of separating the model, the domain, from the user interface. That was, and is, the main goal of MVC.</p><p>The RMR pattern, however, goes beyond that. It tells us how to design our business objects, our domain entities. More than that, it tells us that our domain entities should reflect the delivery mechanism: the HTTP methods.</p><p>This means that this is not just a presentation pattern, it is an architectural pattern as it impacts all layers of the application. It also means that an application built using this pattern is not domain-centric, it is HTTP-centric. Our entities end up having methods reflecting the delivery mechanism instead of domain actions.</p><p>I think it might be possible to use this pattern successfully to build small APIs, but I don’t think it can be used for enterprise applications as I believe that enterprise applications require a Domain Driven Design approach, and therefore a domain-centric software development strategy.</p><p>Furthermore, I totally agree with Anthony Ferrara when he says:</p><blockquote><em>Not to mention that it couples itself to HTTP so tightly that to try to map it to a CLI or GUI interface would be quite difficult.</em></blockquote><blockquote><em>Anthony Ferrara 2014, </em><a href="https://blog.ircmaxell.com/2014/11/alternatives-to-mvc.html#RMR-Resource-Method-Representation"><em>Alternatives To MVC</em></a></blockquote><h3>Sources</h3><p>2008 — Paul James — <a href="http://www.peej.co.uk/articles/rmr-architecture.html">Introducing the RMR Web Architecture</a></p><p>2014 — Anthony Ferrara — <a href="https://blog.ircmaxell.com/2014/11/alternatives-to-mvc.html">Alternatives To MVC</a></p><p><strong>Published</strong> August 31, 2018August 28, 2018</p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/08/31/resource-method-representation/"><em>herbertograca.com</em></a><em> on August 31, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61c7dea1217f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-software-architecture-chronicles/resource-method-representation-61c7dea1217f">Resource-Method-Representation</a> was originally published in <a href="https://medium.com/the-software-architecture-chronicles">The Software Architecture Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Model 1” & “Model 2”]]></title>
            <link>https://medium.com/the-software-architecture-chronicles/model-1-model-2-7baee3852fde?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/7baee3852fde</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[mvc]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Wed, 29 Aug 2018 07:30:13 GMT</pubDate>
            <atom:updated>2018-08-29T08:36:49.967Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a><em>, a </em><a href="https://herbertograca.com/category/development/series/software-architecture/"><em>series of posts about Software Architecture</em></a><em>. In them, I write about what I’ve learned about Software Architecture, how I think of it, and how I use that knowledge. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>Java Server Pages (JSP) is a technology, a scripting language comparable to PHP, ASP, or even Python, that is used to create server-side pages interpreted by the JVM and which can use Java objects.</p><p>The first JSP specifications, published in 1998 by Sun Microsystems, defined two ways of structuring an application so that the presentation logic would be decoupled from the business logic, and even the use cases, in an HTTP request/response paradigm.</p><p>Some consider these “Model 1” and “Model 2” the first tries at adapting the MVC pattern, originally intended for a context of desktop software development, to the web HTTP request/response paradigm.</p><h3>Model 1</h3><p>The first proposal of the <a href="http://www.kirkdorffer.com/jspspecs/jsp092.html#model">JSP spec. v0.92</a>, was to use JSPs as the only presentation artefact, which would contain all the presentation and use-case logic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*JVBVFZAnF2_SDE_0tw_VTQ.jpeg" /><figcaption><em>“Model 1” (</em><a href="http://www.kirkdorffer.com/jspspecs/jsp092.html#model"><em>JSP spec. v0.92</em></a><em>)</em></figcaption></figure><p>This was probably a good enough approach for most use cases at the time as most of the web was composed by simple dynamic pages, not by complex web enterprise applications as we know them today.</p><h3>Model 2</h3><p>The second proposal of how to use JSPs was intended for what was regarded back then as complex web applications. Keep in mind, however, that nowadays web application scale and complexity is quite higher.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*5BPCxoyEzLrV4EhHcDDmjQ.jpeg" /><figcaption><em>“Model 2” (</em><a href="http://www.kirkdorffer.com/jspspecs/jsp092.html#model"><em>JSP spec. v0.92</em></a><em>)</em></figcaption></figure><p>In “Model 2”, an HTTP request would reach a servlet which would interpret the HTTP request, execute some use case logic using Java objects and EJBs (repositories), gather the resulting data and pass that data onto a JSP which would render the page to be sent back to the client. In “Model 2” the JSPs are used exclusively as a template engine.</p><p>In 1999, Govind Seshadri published an article where he maps “Model 2” to MVC:</p><ul><li>the Controller would be the servelet, which controls what should be done with the user request;</li><li>the View would be the JSP, which determines what is shown to the user;</li><li>The Model, both in MVC and in “Model 2” refer to the whole of the Domain Model.</li></ul><h3>My take on these patterns</h3><p>As a curiosity, “Model 2” was actually the first version of “MVC” that I’ve learned, not so long ago.</p><p>In the end, depending on the scope and requirements of the project at hand, both approaches are still usable today.</p><p>That being said, I think that for today’s web enterprise applications we need something better. The problem I see with both patterns is that they don’t respect the Single Responsibility Principle.</p><p>When it comes to “Model 1”, I hope that the mixed concerns are clearly visible to everyone: We would be mixing templating logic with use case logic.</p><p>Regarding “Model 2”, I see both the View and the Controller as belonging to the Presentation layer, however, Govind Seshadri makes it clear that “<a href="https://www.javaworld.com/article/2076557/java-web-development/understanding-javaserver-pages-model-2-architecture.html?page=2"><em>Model 2 architecture should result in the concentration of all of the processing logic in the hands of the controller servlet</em></a>“, although the Domain logic is outside, in Java objects and EJBs.</p><p>This means that the “Model 2” Controller contains the use-case logic, which belongs in the Application layer, not in the Presentation layer.</p><p>For example, if we would want to trigger an existing use case from an event, we would need to duplicate the use-case logic (already existing in a controller) in an event listener and that is a big no-no because it leads to harder maintainability and can lead to inconsistencies throughout the application (bugs).</p><h3>Sources</h3><p>1998 — Sun Microsystems — <a href="http://www.kirkdorffer.com/jspspecs/jsp092.html">JavaServerTM PagesTM — Specification 0.92</a></p><p>1999 — Govind Seshadri — <a href="https://www.javaworld.com/article/2076557/java-web-development/understanding-javaserver-pages-model-2-architecture.html">Understanding JavaServer Pages Model 2 architecture:</a><br> <a href="https://www.javaworld.com/article/2076557/java-web-development/understanding-javaserver-pages-model-2-architecture.html">Exploring the MVC design pattern</a></p><p>2018 — Paul M. Jones — <a href="https://github.com/pmjones/adr/blob/master/MVC-MODEL-2.md">Model View Controller and “Model 2”</a></p><p>2018* — Wikipedia — <a href="https://en.wikipedia.org/wiki/JSP_model_2_architecture">JSP model 2 architecture</a></p><p>*viewed in</p><p><strong>Published</strong> August 29, 2018August 27, 2018</p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/08/29/model-1-model-2/"><em>herbertograca.com</em></a><em> on August 29, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7baee3852fde" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-software-architecture-chronicles/model-1-model-2-7baee3852fde">“Model 1” &amp; “Model 2”</a> was originally published in <a href="https://medium.com/the-software-architecture-chronicles">The Software Architecture Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[More than concentric layers]]></title>
            <link>https://medium.com/@herberto.graca/more-than-concentric-layers-4b8109ebd286?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b8109ebd286</guid>
            <category><![CDATA[explicit-architecture]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[clean-architecture]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Sat, 07 Jul 2018 21:36:58 GMT</pubDate>
            <atom:updated>2018-07-13T13:40:04.625Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a><em>, a </em><a href="https://herbertograca.com/category/development/series/software-architecture/"><em>series of posts about Software Architecture</em></a><em>. In them, I write about what I’ve learned on Software Architecture, how I think of it, and how I use that knowledge. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>In my previous post in this series, I published an infographic that reflects the mental map I use to figure out the relationships between the code units types.</p><p>However, there was something that I always felt it was not very well reflected there, but I just didn’t know how to do it any better: the shared kernel.</p><p>Furthermore, I figured out a few more things, and I’m gonna write about it all in this post!</p><p>Looking at the infographic I published in my last post of this series, we see the shared kernel in the very centre of the diagram, looking like it is within the Domain layer and above the conic sections that represent bounded contexts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/0*aWVrWKGFLLjAZQho" /></figure><p>Despite its location, I did not mean to imply that the shared kernel depends on the remainder of the code nor that the shared kernel is another layer within the Domain layer.</p><h3>What is the shared kernel?!</h3><p>The shared kernel as defined by Eric Evans, the <em>father</em> of DDD, is the code that the development team decides to share between several bounded contexts:</p><blockquote><em>[…] some subset of the domain model that the two teams agree to share. Of course this includes, along with this subset of the model, the subset of code or of the database design associated with that part of the model. This explicitly shared stuff has special status, and shouldn’t be changed without consultation with the other team.</em></blockquote><blockquote><a href="http://ddd.fed.wiki.org/view/shared-kernel"><em>Shared Kernel</em></a><em>, DDD wiki by Ward Cunningham</em></blockquote><p>So basically, it can be any type of code: domain layer code, application layer code, libraries, … whatever.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kT8HkV1QrY-5n5ck1pHJuQ.png" /></figure><p>In the context of the mental map I use, however, I think of it as a subset of that, as specific types of code. In my mental map, the shared kernel contains the domain and application layers code that is shared among bounded contexts in order for communication between bounded contexts to be possible.</p><p>This means, for example, the events that are triggered in one or more bounded contexts and listened to within other bounded contexts. Together with those events we need to share all data types that are used by those events, for example: entities IDs, value objects, enums, etc. Complex objects like entities should not be used be events directly because they can be problematic to serialize/unserialize into/from a queue, so the shared code shouldn’t propagate much.</p><p>Of course that if we have in our hands a polyglot system, composed of micro-services developed in different languages, this shared kernel needs to be descriptive, in json, xml, yaml, or whatever, so that all micro-services can understand it.</p><p>As a result, this shared kernel is completely decoupled from the remainder of the codebase, from the components. This is great, as it means that the components, although coupled to the shared kernel, are decoupled between each other. The shared code is explicitly identified and easily extractable to a separate library.</p><p>This is also quite handy if and when we decide to extract one of the bounded contexts into a micro-service separated from the monolith, we know exactly what is shared and we can simply extract the shared kernel into a library to be installed in both the monolith and the micro-service.</p><p>So to recap, in my mental map, the application core depends on the shared kernel which contains code, from the domain and application layers, that is shared between bounded contexts.</p><h3>When the language is insufficient…</h3><p>So, we have the application code with all its concentric layers, and the application core depends on the shared kernel, which sits below all that code.</p><p>We can also say that all that code depends on the programming language(s) being used, but that is such an obvious fact that we tend to completely ignore it.</p><p>I am bringing it up, however, because of the question “what do we do when the language constructs are not enough?!”. Well, obviously we create those language constructs ourselves and thus compensate for the language flaws. But the important follow up questions I have are “How do we transmit the rationale behind that code existence? Where do we place it? How do we make explicit when and how it should be used?”.</p><p>What I’ve seen, and done myself, is a package called Utils or Commons where that code is placed. But this ends up being a bucked where we dump all the code that we don’t know where to put! All kind of different code, with different purposes and usability (wrapped in an adapter, used directly, …) ends up being dumped there, the package has no conceptual meaning, no consistency, no cohesion, no explicitness, plenty of ambiguity.</p><p>I want to abolish the Utils and Commons packages!</p><p>All packages must have conceptual cohesion! When and how a package is to be used must be explicit! There must be no ambiguity!</p><p>So, for example, if our application has some special way to interact with the CLI, instead of placing it in a namespace ‘Acme/Util/SpecialCli’ we can put it into ‘Acme/App/Infrastructure/Cli/SpecialCli’ which tells us that this package is about the CLI, it is part of the infrastructure of the application ‘App’ of company ‘Acme’. Being part of the App infrastructure also tells us that there is a port, in the application core, to which this package complies to.</p><p>Alternatively, if we see this package as something that the language itself should/could have, we can place it in a namespace that reflects that, for example ‘Acme/PhpExtension/SpecialCli’. This tells us that this package should be seen as part of the language itself, and therefore its code should be used directly in the codebase as any language construct. Of course, however, if another company depends on this package it might be wise for them not to depend directly on it, it is safer to create a port/adapter so they can swap it for something else, but if we own the package we can decide to treat it as part of the language as the risk of needing to swap it for another alternative is not that much. It’s always about the tradeoffs.</p><p>Another example of something I could consider part of the language would be UUIDs in PHP. I can imagine it is not part of the language because there is a new version every once in a while and it would be a maintainability burden, but otherwise it would be a concept generic, widespread and consistent enough to be part of the language.</p><p>So, why not create an implementation of UUIDs and use it as if it was part of PHP itself, just like we use a DateTime object?! As long as we control the implementation, I can see no down side to it.</p><p>What about Doctrine Query Language (DQL) ? (Doctrine is a port of Hibernate into PHP) Can we treat DQL as if it was SQL, Elasticsearch QL, or Mongo QL?</p><h3>Conclusion</h3><p>So in conclusion, I see these 4 major types of code at the macro level, and I think making them explicit in the code organization can be key for us to stop ending up with a big ball of mud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2YYWzE3WnctGW8oE" /></figure><p>What, for me, is an unquestionable truth is that<strong> the architecture will always be there, the only question is: are we in control of it or not?!</strong></p><p>So let’s <strong>explicitly organize the code so it helps communicate the architecture</strong>, follow my mental map fully or partially, follow your own mental map, or follow someone else’s mental map, but please let’s use some consistent reasoning to organise the code and make the project explicitly communicate its architecture using its structure, its code organisation.</p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/07/07/more-than-concentric-layers/"><em>herbertograca.com</em></a><em> on July 7, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b8109ebd286" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deployments with containers]]></title>
            <link>https://medium.com/the-containerization-chronicles/deployments-with-containers-8fba1544e50c?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/8fba1544e50c</guid>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[heroku]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Fri, 15 Jun 2018 08:42:09 GMT</pubDate>
            <atom:updated>2018-06-15T08:58:26.254Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2018/04/02/the-containerization-chronicles/"><em>The Containerization Chronicles</em></a><em>, a series of posts about Containerization. In them, I write about my experiments and what I’ve learned on Containerization of applications. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>Now that we have the project integrated with a Continuous Integration server, where we run the tests and report back to GitHub pull requests with the results of the test run and the coverage, we can deploy our project and make it available on the Internet.</p><p>We will do so with Heroku. I have chosen Heroku because it allows me to perform these experiments for free.</p><p>If you want to jump into the code, <a href="https://github.com/hgraca/explicit-architecture-php/releases/tag/containerisation_chronicles_06">this is the tag for this post</a>.</p><p>Heroku will keep our application up on the Internet for 30 minutes after the last request to it. After those 30 minutes, it will go to “sleep” until a request to it is made. At that point, it will take some 20 seconds to wake up and gives us a response, but that’s fine, these are just experiments.</p><p>After creating our account on Heroku, we need to create a <em>pipeline</em>. We will call it ppln-explicit-arch-php-sfn. By default, a pipeline will have two environments:</p><ul><li><strong><em>staging</em></strong>: where our Product Owner (PO) can test the application manually;</li><li><strong><em>production</em></strong>: where <em>all our users</em> can reach the application.</li></ul><h3>The workflow with Scrutinizer and Heroku</h3><p>What we would like to have is a workflow in this format:</p><ol><li>Developers work on a task locally;</li><li>The feature is regularly pushed to GitHub for safekeeping;</li><li>When the task is ready, the developer makes a pull request (PR) to master;</li><li>The PR triggers the tests in Scrutinizer;</li><li>If the tests are green, it triggers a deploy to a Heroku app on <em>staging</em>;</li><li>The PR is reviewed;</li><li>The staging feature is accepted by the PO;</li><li>The developer merges the PR to the master branch;</li><li>The merge to master triggers the tests in Scrutinizer again (because new changes to the master branch, made in the meantime, might break our code);</li><li>If the tests are green, Scrutinizer triggers a deploy to <em>production</em>.</li></ol><p>For this, we need to limit Scrutinizer to monitoring only the master branch, so that only PRs and merges to master would trigger test runs and deployments. Otherwise, we would have Scrutinizer running tests and maybe deployments when it’s not needed, eventually creating a job queue and slowing down the whole process.</p><h3>A challenge: Scrutinizer limitation as a CI</h3><p>Unfortunately, <strong>Scrutinizer doesn’t trigger deployments when we make a PR</strong>, although it does trigger the tests run.</p><p>This means that we can not use PRs for deployment to staging, so the alternative is to use pushes to specific branches in order to trigger deployments to staging. We will make it so that whenever we make a push to a branch starting with stg-, Scrutinizer will deploy to a staging app on Heroku.</p><p>And this, however, means two things:</p><ol><li>We don’t need Scrutinizer to monitor PRs;</li><li>We need Scrutinizer to monitor all branches, instead of only <em>master</em>, which will make Scrutinizer run the tests for all branches we create on GitHub 😦 .</li></ol><p>This is, of course, a workaround. It’s not nice. I don’t particularly like it. But I also don’t want to put time investigating another CI engine at this moment, although I will surely do it at a later time.</p><p>So the actual workflow will be:</p><ol><li>Developers work on a task locally;</li><li>The feature is regularly pushed to GitHub for safekeeping;</li><li>When the task is ready, the developer makes a pull request (PR) to master;</li><li>The PR triggers the tests in Scrutinizer;</li><li>The PR is reviewed;</li><li><strong>When the review is green, the developer pushes to a branch starting with</strong> <strong>stg-</strong>,<strong> which triggers the tests and a deployment to staging;</strong></li><li>The staging feature is accepted by the PO;</li><li>The developer merges the PR to the master branch;</li><li>The merge to master triggers the tests in Scrutinizer again (because new changes to the master branch, made in the meantime, might break our code);</li><li>If the tests are green, Scrutinizer triggers a deploy to <em>production</em>.</li></ol><h3>Scrutinizer config</h3><p>For the workflow above we need to configure Scrutinizer as follows, on their website:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PZRe5cAlmMSv2UpR" /></figure><p>As you can see above, we are not tracking PRs nor any specific branches.</p><p>In our .scrutinizer.yml config we will need to add a snippet so that Scrutinizer knows we want to deploy to Heroku:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/0*VluGIlAltBIQepFc" /><figcaption><em>.scrutinizer.yml</em></figcaption></figure><p>This tells Scrutinizer that every time there is a merge/push to a branch starting with stg- it needs to run the command make deploy_stg-ci, and whenever there is a merge/push to master, it needs to run the command make deploy_prd-ci.</p><h3>The deployment script</h3><p>Now we need to create those commands in our <em>Makefile</em>. I started by creating those commands directly in the Makefile, but it turned out to be too complex to just put them in the Makefile, so I created a reusable deployment script in bash and call it from the Makefile.</p><p>The deployment script goes like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/0*o9A2_L4PJ-UQIwtd" /><figcaption><em>bin/deploy</em></figcaption></figure><p>The script receives two arguments, the first is the stage of the Heroku app in the pipeline (either <em>staging</em> or <em>production</em>), and the second is the Heroku app name, which will be the same as the branch being deployed.</p><p>The script will get the variables in a brown background from the environment, so we need to make sure the environment where the script runs has them in the environment. Since this script will run in the CI, we need to set those environment variables in Scrutinizer, and since we don’t want to have those variables in the repository (for security reasons and so that they can be changed without redeploying the app) we will add them in the Scrutinizer web config, as opposed to the yaml config:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BCX2GRZAt8gYjUub" /></figure><p>The Makefile additions are trivial:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/0*5Omk7g7UnTa0_-62" /><figcaption><em>Makefile</em></figcaption></figure><h3>Running the application in Heroku</h3><p>If we take a look at the deployment script above, we will notice that when we deploy to Heroku, we simply send a docker image to the Heroku registry.</p><p>The only thing Heroku does is start the container, so whatever the container entry point is, that’s what will be running. This seems sensible, but Heroku brings up the container making our application listen on a random port every time and makes that port available in the environment.</p><p>Our image current entry point is set in the production image dockerfile:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/0*PsW8La28p-fz1bDW" /><figcaption><em>build/container/prd/app.dockerfile</em></figcaption></figure><p>As we can see, the port where our application listens is hardcoded there, and we need to make it be in sync with what Heroku puts in the environment variable.</p><p>As stated in the <a href="https://docs.docker.com/engine/reference/builder/#environment-replacement">Docker documentation</a>, we can simply do this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/0*hmQnIBjFy-IRFaOD" /><figcaption><em>build/container/prd/app.dockerfile</em></figcaption></figure><p>Unfortunately, I couldn’t make this work!! (if you know how to make it work, please let me know!!) So I needed to create a script that the dockerfile will reference and that script will be able to get the port from the environment.</p><p>The script to bring the application up is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/0*2tZIXN2OwSPUk4ue" /><figcaption><em>bin/up.sh</em></figcaption></figure><p>And we change the dockerfile accordingly:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/0*e9oTgBu5SSKX_6MR" /><figcaption><em>build/container/prd/app.dockerfile</em></figcaption></figure><p>This is it.</p><p>Please, feel free to share your thoughts and/or ways to improve this.</p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/06/15/deployments-with-containers/"><em>herbertograca.com</em></a><em> on June 15, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8fba1544e50c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-containerization-chronicles/deployments-with-containers-8fba1544e50c">Deployments with containers</a> was originally published in <a href="https://medium.com/the-containerization-chronicles">The Containerization Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Continuous Integration with containers]]></title>
            <link>https://medium.com/the-containerization-chronicles/continuous-integration-with-containers-2c22a86021b?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c22a86021b</guid>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Thu, 03 May 2018 08:01:09 GMT</pubDate>
            <atom:updated>2018-05-03T08:28:38.360Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2018/04/02/the-containerization-chronicles/"><em>The Containerization Chronicles</em></a><em>, a series of posts about Containerization. In them, I write about my experiments and what I’ve learned on Containerization of applications. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>Now that we have some containers in place for <em>development</em>, we can integrate with a CI server. I chose Scrutinizer CI because of the code analysis it makes, the fact that it is also a CI server it’s a plus. However, as a CI, it has some limitations, which I will talk about later on when integrating with Heroku, so we will most likely move to another CI engine later in the future while keeping Scrutinizer only for the code analysis but, for now, this is all we need.</p><p>To set up the CI, we will:</p><ol><li>Containerize an environment to run the tests in the CI</li><li>Configure Scrutinizer</li><li>Add the scrutinizer badges to the README.md</li><li>Integrate Scrutinizer with GitHub</li></ol><p>If you want to jump right into the code, <a href="https://github.com/hgraca/explicit-architecture-php/releases/tag/containerisation_chronicles_04">this is the tag on GitHub</a>.</p><h3>1. Containerize an environment to run the tests in the CI</h3><p>In the <em>production</em> container, we don’t want to have code that is not used. Why not? Because the container image will be bigger, the attack surface will also be bigger, and it’s just plain dirty.</p><p>Ideally, the container used to run the tests is the same as the one used in production, which is called something like “testing/production parity”. It is the only way to guarantee that a container will perform as expected in production. However, in order to run the unit/integration/functional tests, we need them in the container, as well as all the needed dependencies to run them… So the idea is that we create a container to run the tests, based on the production container but where we add all the tests and their dependencies.</p><p>Unfortunately, this means that the tests run in a project context that is not the same as the context that runs in production. Testing/production parity is broken.</p><p>The danger here is that, for example, our production application will use a dependency which is mistakingly installed as a test dependency. In such a case, the tests will be green but in <em>production</em> the dependency will be missing.</p><p>The solution for that is to have acceptance tests running outside the container against the production container, simulating a user on a browser. The project, however, does not have acceptance tests so I had to create some.</p><p>I created three acceptance tests simulating an anonymous user, a logged-in user and an admin user. The tests themselves are not very elegant, they should probably be broken up into several more tests and not depend on any hardcoded variables, but they are also not the main point of this post, I just want you to know that you shouldn’t take that code as a good example, there are better ways of doing acceptance tests. For the purpose of this blog post, those tests will illustrate perfectly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/415/0*f0boCUr8EdtTclgc." /><figcaption><em>build/container/ci/app.dockerfile</em></figcaption></figure><p>The dockerfile for the CI environment is based on the production image and the only thing it does is adding the tests and its dependencies to the image. This could also be done with multi-stage builds, but I will leave that for a later post where I will experiment with it and analyze the pros and cons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/0*gl6SgeE1LEaJz2EE." /><figcaption><em>build/container/ci/docker-compose.yml</em></figcaption></figure><p>The docker-compose file is also pretty straightforward. When compared to the docker-compose files explained before, we just change the image and container names, the path to the dockerfile and a few environment variables.</p><h3>2. Configure Scrutinizer</h3><p>Scrutinizer is configured in two places, the configuration file in our project root and on their website. In the configuration file, we set up the actual configuration, while in the website we mostly set the environment variables.</p><p>This is how the configuration on the website looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5efy6Aj5e0wlgiNi." /></figure><p>The configuration file is more interesting:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/599/0*4GmUID8db1L9Mr4u." /><figcaption><em>.scrutinizer.yml</em></figcaption></figure><p>Many of those configurations are self-explanatory. It is, however, relevant to mention that scrutinizer already has some default commands it runs to set up the dependencies and to run the tests.</p><p>As you surely noticed above, we used a new make command in the scrutinizer config. These were the relevant changes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/0*CBMZb8Knw72ZXfIB." /><figcaption><em>Makefile</em></figcaption></figure><p>The unit/integration/functional tests will run inside the test container, but the acceptance tests will run on the scrutinizer host, hitting the production container (without any extra tests inside nor dependencies).</p><p>We don’t want to use the default command to run the tests because we run them inside the container, so we override it, use our make command and we specify where scrutinizer can find the coverage report.</p><p>A bit lower in the scrutinizer config file, we have another important configuration: the build_failure_conditions directive. It tells Scrutinizer when to fail the build even if the tests are all green. In our case, we fail the build if any new issue exists or if the coverage drops more than 5%.</p><h3>3. Add the scrutinizer badges to the README.md</h3><p>We want to show the world how cool, well coded and well tested our application is, so we want to add some badges to it. It is fairly simple, we just need to add the following snippets to the README.md:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/0*_D00P056gsx5n7Fr." /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/0*vwY9KrOSIE4K5yTx." /><figcaption><em>README.md</em></figcaption></figure><h3>4. Integrate Scrutinizer with GitHub</h3><p>The only thing left to do is integrate Scrutinizer with GitHub, so that when the tests fail, we will not be allowed to merge to master.</p><p>This is done in the GitHub repository configuration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/0*ClYdE3FnPEYbNhXA." /></figure><p>And it will yield something like this in case the tests fail:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/0*o0R1Xqam-cIBzFq8." /></figure><p>This is it! the integration with Scrutinizer CI is done!</p><p>Please, feel free to share your thoughts and/or ways to improve this.</p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/05/03/continuous-integration-with-containers/"><em>herbertograca.com</em></a><em> on May 3, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c22a86021b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-containerization-chronicles/continuous-integration-with-containers-2c22a86021b">Continuous Integration with containers</a> was originally published in <a href="https://medium.com/the-containerization-chronicles">The Containerization Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Containerize dev, tst and prd]]></title>
            <link>https://medium.com/the-containerization-chronicles/containerize-dev-tst-and-prd-1320cacc5d31?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/1320cacc5d31</guid>
            <category><![CDATA[php]]></category>
            <category><![CDATA[symfony]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[symfony4]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Wed, 18 Apr 2018 08:39:01 GMT</pubDate>
            <atom:updated>2018-04-18T08:44:28.477Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2018/04/02/the-containerization-chronicles/"><em>The Containerization Chronicles</em></a><em>, a series of posts about Containerization. In them, I write about my experiments and what I’ve learned on Containerization of applications. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>After cleaning up the Symfony demo project a bit, we are ready to do some containerization…</p><p>I will talk about:</p><ol><li>Logging</li><li>Containerizing dev, tst, and prd</li><li>Run and stop the containers</li><li>Remove database from VCS</li><li>Integrate with PHPStorm</li></ol><p>If you want to jump right into the code, <a href="https://github.com/hgraca/explicit-architecture-php/releases/tag/containerisation_chronicles_03">this is the tag on GitHub</a>.</p><h3>1. Logging</h3><p>The containers, while running, will output anything that is written to stdout.</p><p>In order to get the logs from the container, we will simply make sure the application writes its logs to stdout. This will make it super easy for us to see the logs while developing but, for production, it will also make it super easy to aggregate the logs of all containers and direct them into a centralized location. As I understand at this moment, Kubernetes has some functionality to do just this! I will get to that sometime in a later post.</p><p>For now, in our case we just need to add a bit of configuration to our application logger, Monolog:</p><p>In the Monolog configuration for the development environment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*P-YoGrnl-S6Jg-3Q." /></figure><p>In the Monolog configuration for the production environment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KJoCaT-r0Tb4u0jq." /></figure><h3>2. Containerizing dev, tst, and prd</h3><p>The demo application is fairly simple and runs only with PHP, on PHP embedded web server and an SQLite file as the database. For this reason, we would be able to containerize the application using only Docker, however, since we want to extend the application to use an http server and a database server in the future, we will use docker compose from the start.</p><h3>2.1 The base container image</h3><p>We will be building a new production container image every time we release a new version of our application. However, the OS dependencies our production container has will be much more stable, they will change at a much slower pace than our application.</p><p>So, we will create a base container image with all those stable OS dependencies, so that we don’t need to install them every time we create a new production image.</p><p>This docker compose file is very simple, it only defines the name of the image to be built and the docker file to be used to build it.</p><p>The dockerfile is more interesting. We base our image on the default PHP7.1 Alpine image, and we install all the OS related dependencies.</p><p>We install all the dependencies in one RUN command, so that it will only create one layer in the image, making it smaller. We start by updating the apk sources and ask it to install the dependencies.</p><p>We use <em>apk</em> two times, the first time to install the dependencies we can remove at the end of the process and the second time to install the dependencies we need to keep in the image.</p><p>The --no-cache flag tells <em>apk</em> to not cache the downloaded applications and the --virtual flag tells <em>apk</em> to tag the following dependencies with the tag build-dependencies, which will be very convenient at the end of the process to remove them in one go.</p><p>After installing all dependencies, we make sure we remove all cache and unneeded dependencies.</p><h3>2.2 The production container image</h3><p>The production image will be a self-contained image with all that is needed to run the application in production. This means it will contain all the application code and all its dependencies. Furthermore, the container entry point will be the command that brings the application up and makes it ready to receive requests.</p><p>As we are using SQLite as the database engine, the SQLite file will be a new one in every deployment. This is fine for now, we will add a Mysql container at some point in the future.</p><p>As we can see on the top, the docker image is based on the base image explained above.</p><p>The ARG lines tell docker that the variables APP_ENV and DATABASE_URL are required at the time of building the image, although this does not make it a requirement for running the container.</p><p>As, by default, this image is supposed to run in production we set the environment variable ENV to prd.</p><p>We copy, into the image, only what is needed to run the application in production. We take special attention to exclude the <em>tests</em> and <em>build</em> folders as in time they will add a considerable amount of space, and we set the workdir to the root of the application inside the image. Although the dockerfile is in a nested folder, we only need to specify the . as the source because this will be run by docker compose, which will set the context to the root of the project.</p><p>So far we just copied the application into the image, but now we also need to prepare it to run, which means install all its dependencies, for which we use a make command that we added to our makefile, and create the database that will be used. We also set the var folder permissions to 777 so that if we run the container with a non-root user we can still use that folder for caching, etc. In the end, we cleanup the composer cache because we don’t need it inside the image, so this way we make the image smaller.</p><p>We also copy our custom php config into the image, and we shouldn’t change it for other environments images so that we develop and test with the same config.</p><p>Finally, we set up the command to run as the default entry point to the container, which in this case it’s starting the php embedded http server to listen on port 8000.</p><p>The EXPOSE instruction does not have any practical reflection, its basically for documentation, let’s say it’s a good practice.</p><p>In the docker-compose file we set the version to 3 because we don’t need any higher and also, at this moment, PHPStorm is not compatible with higher versions.</p><p>We set the name of the image that we will create from this docker compose file, as well as the container name, and the working dir when the container is up. On the build section, we need to set the context to 3 folders above the folder of this file because that’s the project root, and then we need to reference the dockerfile from the project root. We also need to specify the ARG variables required by the dockerfile to build the image.</p><p>The remaining relevant configuration is the ports. We map the host port 80 to the container port 8000 because port 8000 is where the application is listening inside the container and port 80 is the default port used by http. This way, on the host we can access the application using <a href="http://localhost/">http://localhost</a> without specifying any port. If we would already be using the host port 80 for some other application, we would need to use another port instead, for example port 8080 and then we would reach the application as <a href="http://localhost:8080/">http://localhost:8080</a>.</p><h3>2.3 The development container image</h3><p>The development image will be based on the production image. It will be used during development, as an always up server that we can use to test the application on a browser. For this, unlike the production image, it will have a volume mounted where the application code resides so that the container can access the code under development.</p><p>Part of what we can see in the dockerfile was already explained before so I won’t repeat myself. What is relevant at this point is that we want to have xdebug running while developing (we won’t have it turned on during testing, we will see that later) so we install xdebug using pecl. As it happens, to install xdebug we need g++, so we need to install it and then remove it at the end of building the image.</p><p>We also copy our xdebug config into the image, so that no matter how we bring the container up, we always have xdebug configured there.</p><p>Most of what is in the dev docker-compose file is the same as before.</p><p>The relevant changes are that we need to have a tty available in the container so that we can run and test CLI commands and also because some of Symfony code used in dev relies on it. We also create a few volumes so that we can share files with the processes that run inside the container:</p><ul><li>the whole codebase;</li><li>the composer cache, so we can update dependencies from within the container;</li><li>the php configuration;</li><li>the xdebug configuration.</li></ul><h3>2.4 The testing container image</h3><p>The testing image will be used during development, in this case, to run the test suite against the code under development. Just like the development image, it will have a volume where the code under development will be accessible to the container.</p><p>This container is needed so that running the test suite doesn’t inadvertently conflict with the running application, which could occur if we were only using the development image.</p><p>For the testing environment, we don’t even have a dockerfile, the image is exactly the same as the dev image, with some subtle exceptions in the docker compose file:</p><ul><li>The container name is different so that it will not clash with the dev container name, as we want to have the dev container up and run the tests in the test container simultaneously;</li><li>The php.ini file is in the same folder as the testing docker compose file so we can easily tweak the configurations, for example, the available memory (keep in mind, though, that the config should be the same as in production so this should be used only for experimentation);</li><li>The xdebug config file is also in the same directory as the testing docker compose file so that we can enable or disable it at will before running the tests, which is useful if we want to debug the tests.</li></ul><h3>3. Remove the database from VCS</h3><p>This demo app is currently keeping the database in the VCS, as a sqlite file, which is not such a nice practice because everyone is constantly forced to download the DB from VCS, and roll back the changes every time after running the tests.</p><p>However, since from now on we use the makefile test, test_cov and up commands to test and start the application, we don’t need to keep the database in the VCS anymore because those commands will generate the DB if it is not present, so we remove those files from VCS.</p><h3>4. Run and stop the containers</h3><p>While we are developing our application we want an easy way of running and testing it. We don’t want to lose our focus trying to remember docker commands, or having to remove several stopped containers every once in a while.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/117/0*a6h16sqtD1vsrse2." /></figure><p>Furthermore, the containers should be ephemerous, so we can safely remove them once we have used them.</p><p>In order to improve our DevEx, we create a run and a stop commands, and further abstract them behind our makefile:</p><ul><li>The run command:</li><li>allows us to start the container and related containers, and then run other commands against it;</li><li>With no arguments, the container will be started and kept alive for 1h;</li><li>With arguments, if the container is running it will send the commands to it and leave it running;</li><li>With arguments, if the container is NOT running it will start it, send the commands to it and terminate it.</li><li>The stop command will simply stop all containers related to the docker compose file.</li></ul><p>And then we further abstract them behind our makefile with a shorthand commands, like for example:</p><pre>cs-fix:<br>    ENV=&#39;tst&#39; ./bin/run php vendor/bin/php-cs-fixer fix --verbose</pre><pre>db-setup:<br>    ENV=&#39;dev&#39; ./bin/run make db-setup-guest</pre><pre>dep-update:<br>    ENV=&#39;dev&#39; ./bin/run composer update</pre><pre>test:<br>    - ENV=&#39;tst&#39; ./bin/stop # Just in case some container is left over<br>    ENV=&#39;tst&#39; ./bin/run <br>    ENV=&#39;tst&#39; ./bin/run make db-setup-guest <br>    ENV=&#39;tst&#39; ./bin/run php vendor/bin/phpunit $(MAKE) cs-fix <br>    ENV=&#39;tst&#39; ./bin/stop</pre><pre>test_cov: <br>    ENV=&#39;tst&#39; ./bin/run phpdbg -qrr vendor/bin/phpunit --coverage-text --coverage-clover=${COVERAGE_REPORT_PATH}</pre><h3>5. Integrate with PHPStorm</h3><p>Integrating with PHPStorm becomes straightforward.</p><p>We configure the server so that we can debug from the browser:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Isn2xnBc5ZAtSpoy." /></figure><p>We configure the CLI runner so that we can run the tests suites:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BiK6KbqWxAFwHYDP." /></figure><p>And finally we configure a test run:</p><p>I trust you are able to figure out how to run the tests and the debugger.</p><p>Well, if you reached this point in the post, congratzz!! 🙂</p><p>Please, feel free to share your thoughts and/or ways to improve this.</p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/04/18/containerize-dev-tst-and-prd/"><em>herbertograca.com</em></a><em> on April 18, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1320cacc5d31" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-containerization-chronicles/containerize-dev-tst-and-prd-1320cacc5d31">Containerize dev, tst and prd</a> was originally published in <a href="https://medium.com/the-containerization-chronicles">The Containerization Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cleaning up the demo project]]></title>
            <link>https://medium.com/the-containerization-chronicles/cleaning-up-the-demo-project-23dc18bff17f?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/23dc18bff17f</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[php]]></category>
            <category><![CDATA[symfony4]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[symfony]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Fri, 13 Apr 2018 10:52:58 GMT</pubDate>
            <atom:updated>2018-04-18T08:01:42.518Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2018/04/02/the-containerization-chronicles/"><em>The Containerisation Chronicles</em></a><em>, a series of posts about Containerisation. In them, I write about my experiments and what I’ve learned on Containerisation of applications. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>In order to start making experiments with containerization of applications, I am going to be using the <a href="https://github.com/symfony/demo">Symfony demo project</a>, which is a small blog where one can view and create some blog posts. Although it is a very small project, and therefore does not reflect the complexity of an enterprise cloud application, I feel it is flexible enough to experiment containerization practices, especially because I want to start small.</p><p>However, there are some things I don’t like about the project, so I start by doing some cleanup:</p><ol><li>Add roave/security-advisories</li><li>Remove simple-phpunit, update to phpunit 7 and DAMA bundle 5</li><li>Disable the profiler during test runs, so the tests run a bit faster</li><li>Type hint all the things</li><li>Change namespace from `App` to `Acme\App` to better comply with PSR-4</li><li>Add a Makefile so we abstract our DevEx and automation from the tools</li></ol><p>If you want to jump right into the code, <a href="https://github.com/hgraca/explicit-architecture-php/releases/tag/containerisation_chronicles_02">this is the tag on GitHub</a>.</p><h3>1. Add roave/security-advisories</h3><p>The <a href="https://github.com/Roave/SecurityAdvisories">roave/security-advisories</a> package <em>ensures that your application doesn’t have installed dependencies with known security vulnerabilities</em>.</p><p>As mentioned in the project GitHub, <em>the checks are only executed when adding a new dependency via composer require or when running composer update: deploying an application with a valid composer.lock and via composer install won’t trigger any security versions checking</em>. So basically this only makes sense to install in the development requirements of the application, although at the moment PHPStorm still asks us to install that package in the default application dependency requirements.</p><p>Of course, this project will never be a real “production” application so maybe this could be skipped, but it is a good practice to do so and my OCD forces me to do it!</p><p>So, we add it to <em>composer.json</em>:</p><p>And run <em>composer update</em>…</p><p>For some reason, it also removed the <em>phpunit.xml.dist</em>, but I rolled it back before merging so we keep the project as unchanged as possible.</p><h3>2. Remove simple-phpunit, update to phpunit 7 and DAMA bundle 5</h3><p>This demo project comes with <a href="http://symfony.com/doc/current/components/phpunit_bridge.html">simple-phpunit</a> which, at the time of this writing, gives the following features:</p><blockquote><em>The PHPUnit Bridge provides utilities to report legacy tests and usage of deprecated code and a helper for time-sensitive tests.</em></blockquote><blockquote><em>It comes with the following features:</em></blockquote><blockquote>- Forces the tests to use a consistent locale ( C );<br>- Auto-register class_exists to load Doctrine annotations (when used);<br>- It displays the whole list of deprecated features used in the application;<br>- Displays the stack trace of a deprecation on-demand;<br>- Provides a ClockMock and DnsMock helper classes for time or network-sensitive tests.<br>- Provides a modified version of PHPUnit that does not embed symfony/yaml nor prophecy to prevent any conflicts with these dependencies.</blockquote><blockquote><a href="http://symfony.com/doc/current/components/phpunit_bridge.html"><em>The PHPUnit Bridge</em></a></blockquote><p>However, I see the following disadvantages:</p><ul><li>At the first run, it will start downloading dependencies which is weird because that is the role of composer and it brings us problems later when running inside the container because ideally the user running commands inside the container should only have permissions to change the contents of the var folder;</li><li>Will use the phar version of PHPunit, which makes it not be managed by composer;</li><li>Because it is a wrapper around PHPUnit, we can only upgrade to the latest version of PHPUnit is that wrapper supports it;</li><li>I could not integrate it with PHPStorm;</li><li>There is an <a href="https://github.com/sebastianbergmann/phpunit/issues/2015">issue</a> that happens when the code under test shares dependencies with PHPUnit but requires different versions than the ones bundled in the PHAR;</li></ul><p>Nevertheless, we really want to get as many of those features as possible and, because the Symfony devs are cool, we can actually get those features (or at least some of them) by using the <em>PHPUnit Bridge</em> without using simple-phpunit!</p><p>So we add the PHPUnit v7 dependency and upgrade the dama/doctrine-test-bundle to version 5, but we leave in the <em>PHPUnit Bridge</em>:</p><p>To make PHPUnit get the best of the <em>PHPUnit Bridge</em>, we add a listener to phpunit.xml.dist:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HXjJN_pFz4LfOPiz." /></figure><p>We also add the filter section so that we can get the code coverage report from PHPUnit.</p><p>After running composer update, we get some extra files, including one in <em>./bin/phpunit</em>, however, this one is the executable that runs simple-phpunit, so I will not even commit it. What we want to use is the phpunit executable at <strong><em>./vendor/bin/phpunit</em></strong>.</p><p>We also update the README.md file to reflect the executable we should use to run PHPUnit.</p><h3>3. Disable the profiler during test runs, so the tests run a bit faster</h3><p>Very recently, at work, we realized that the Symfony Framework Extra Bundle has the profiler enabled by default during test runs. This has a performance impact which is <strong>negligible in a small project</strong> but it is <strong>significant in a big project</strong> like the one I am currently working on, which came down to <strong>several minutes</strong> difference when running the functional and acceptance tests.</p><p>These were the changes made to the profiler configuration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*N0z3sKptkgI8ScVz." /></figure><p>We turn it off altogether and set the collect setting to true. This way, if we want to use it during test runs we just need to turn it on, and we don’t need to add any code to start the profiler. Otherwise, we would need to do something like $profiler-&gt;enable(); in our code, to start the profiler.</p><p>Looking at the configuration above, one might think that the default configuration (enabled: true; collect: false) ends up being the same since the profiler will only do something if we add some code to start the profiling, however, I tried it a few times and always got some performance improvement.</p><h3>4. Type hint all the things</h3><p>Since this project evolved from PHP5, there are plenty of type hints missing so we add them to prevent bugs and satisfy our OCD.</p><p>We also prefix the native function calls with the \ which tells PHP that the function we want to call belongs in the global namespace. This prevents PHP from searching for the function in all namespaces until it finds it, which can give us some minor performance improvements.</p><p>We did this two things in only one commit, which is hurting my OCD, but I just wanted to get it done and don’t anticipate a problem coming out of it.</p><h3>5. Change namespace from `App` to `Acme\App` to better comply with PSR-4</h3><p>The <a href="https://www.php-fig.org/psr/psr-4/">PSR-4: Autoloader</a>, <em>describes a specification for autoloading classes from file paths. It is fully interoperable and can be used in addition to any other autoloading specification, including PSR-0. This PSR also describes where to place files that will be autoloaded according to the specification</em>.</p><p>Part of it is about the classes fully qualified namespace, and it goes like this:</p><blockquote><em>2. A fully qualified class name has the following form:</em></blockquote><blockquote><em>\&lt;NamespaceName&gt;(\&lt;SubNamespaceNames&gt;)*\&lt;ClassName&gt;</em></blockquote><blockquote><strong><em>2.1 — The fully qualified class name MUST have a top-level namespace name, also known as a “vendor namespace”.<br></em></strong><em>2.2 — The fully qualified class name MAY have one or more sub-namespace names. <br>2.3 — The fully qualified class name MUST have a terminating class name. <br>2.4 — Underscores have no special meaning in any portion of the fully qualified class name. <br>2.5 — Alphabetic characters in the fully qualified class name MAY be any combination of lower case and upper case.<br>2.6 — All class names MUST be referenced in a case-sensitive fashion.</em></blockquote><blockquote>PHP FIG — <a href="https://www.php-fig.org/psr/psr-4/">PSR-4: Autoloader</a></blockquote><p>In this fourth step, we are concerned with point 2.1, which mentions the “<em>vendor namespace</em>“. This vendor namespace can be a company namespace or our personal namespace so that if there are several projects with the same name their namespace will not clash.</p><p>Our project namespace is simply App\, which is the application name, so we are missing the vendor namespace. So we change the project namespace from App\ to Acme\App\.</p><h3>6. Add a Makefile so we abstract our DevEx and automation from the tools</h3><p>Makefiles were initially created to help compile C++ into executable programs, but they can be used for other purposes as well. In simple terms, it is a file with shell commands that can be called as an argument to <em>make</em>. Nevertheless, it does have more capabilities that we will not need to use, like keeping track of source files changes.</p><p>We will use it as:</p><ul><li>An <strong>entry point</strong> to the application development and automation commands:<br>We add in it the commands we use during development and in the CI/CD servers setup, for example, the commands to install and update dependencies.</li><li>An <strong>abstraction layer</strong> over the tools used under the hood:<br>The commands must not refer to the specific tools used, so that we can switch the tools used and keep using the same commands, both during development and in the automation tools configuration, for example, instead of naming a command make composer-update we should name it something like make update-dependencies, this way even if we switch from composer to some other tool, the command name will still make sense.</li></ul><p>Sure, we could use a shell file for this as well, but the makefile forces a format on us and limits us on the complexity of the logic we can put in it. In other words, it helps us not to get it messy.</p><p>It is also important to name and organize the commands according to some convention, as to make them predictable and easy to find.</p><p>The convention used in this case is subject-action-environment, and organize them alphabetically. This will keep commands names meaningful and close together.</p><p>At this point, the commands added are as follows:</p><ul><li>cs-fix<br>Fixes the code standards</li><li>db-setup<br>Creates the database</li><li>dep-clearcache<br>Removes the dependencies cache</li><li>dep-install<br>Installs all the dependencies, for development</li><li>dep-install-prd<br>Installs all the dependencies, for production (optimized and no dev dependencies)</li><li>dep-update<br>Updates the versions of the dependencies</li><li>test<br>Tests the code base</li><li>test_cov<br>Tests the codebase and generates the code coverage report</li><li>up<br>Starts the application so that it can start receiving requests</li></ul><h4>6.1 The Makefile.custom.dist file</h4><p>The resulting command names might at times not be the easiest to remember, and the developers in the team might want to use their own commands, so we also add a <em>Makefile.custom.dist</em> file which the developers can copy into a <em>Makefile.custom</em> file, put there their own commands or aliases and use it as if they were in the main <em>Makefile</em>.</p><p>That’s it for today!</p><p>Please, feel free to share your thoughts and/or ways to improve this.</p><h3>Resources</h3><p><a href="https://github.com/hgraca/explicit-architecture-php/commits/containerisation_chronicles_01">Commits referenced by this post, on GitHub</a></p><p><em>Originally published at </em><a href="https://herbertograca.com/2018/04/13/cleaning-up-the-demo-project/"><em>herbertograca.com</em></a><em> on April 13, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23dc18bff17f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-containerization-chronicles/cleaning-up-the-demo-project-23dc18bff17f">Cleaning up the demo project</a> was originally published in <a href="https://medium.com/the-containerization-chronicles">The Containerization Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Containerization Chronicles]]></title>
            <link>https://medium.com/the-containerization-chronicles/the-containerization-chronicles-aaf2feef2f55?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/aaf2feef2f55</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Mon, 02 Apr 2018 11:33:04 GMT</pubDate>
            <atom:updated>2018-04-02T11:49:13.302Z</atom:updated>
            <content:encoded><![CDATA[<p>While working on the 18th post of <a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a>, which is about implementing a proof of concept, I picked up the Symfony demo app and started refactoring away…</p><p>I started by cleaning up the project a bit and adding some functionality I like to have for better DevEx (development experience), but after some 20 commits I got annoyed with it and decided to try to get the environment as best as I could.</p><p>More or less at the same time, at work, we are currently moving to containers, and Kubernetes, and this would be a nice project to start diving into that.</p><p>So here it is, “The Containerization Chronicles”, a series of posts where I will be describing my experiments and learnings about the containers world.</p><p>I will be using the same repository as I will use for <a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a>, so you will be able to access all the code <a href="https://github.com/hgraca/explicit-architecture-php">there</a>.</p><h3>Posts in The Containerization Chronicles:</h3><ol><li>The Containerization Chronicles</li><li>Cleaning up the demo project</li><li>Containerize dev and “production”</li><li>Integrate with Scrutinizer CI</li><li>Integrate with CodeCov</li><li>Integrate with Heroku</li><li>Add an Nginx container</li><li>Replace the Nginx container for httpd</li><li>Replace the httpd container for httpd+mod_php</li><li>Make a benchmark on the previous 3 configurations</li><li>Add a persistence container</li><li>Start diving into Kubernetes… No idea what I’m gonna do here…</li><li>…</li></ol><p><em>Originally published at </em><a href="https://herbertograca.com/2018/04/02/the-containerization-chronicles/"><em>herbertograca.com</em></a><em> on April 2, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aaf2feef2f55" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-containerization-chronicles/the-containerization-chronicles-aaf2feef2f55">The Containerization Chronicles</a> was originally published in <a href="https://medium.com/the-containerization-chronicles">The Containerization Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DDD, Hexagonal, Onion, Clean, CQRS, … How I put it all together]]></title>
            <link>https://medium.com/the-software-architecture-chronicles/ddd-hexagonal-onion-clean-cqrs-how-i-put-it-all-together-f2590c0aa7f6?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2590c0aa7f6</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[explicit-architecture]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Thu, 16 Nov 2017 14:30:54 GMT</pubDate>
            <atom:updated>2017-12-27T08:01:02.721Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a><em>, a </em><a href="https://herbertograca.com/category/development/series/software-architecture/"><em>series of posts about Software Architecture</em></a><em>. In them, I write about what I’ve learned on Software Architecture, how I think of it, and how I use that knowledge. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>After graduating from University I followed a career as a high school teacher until a few years ago I decided to drop it and become a full-time software developer.</p><p>From then on, I have always felt like I need to recover the “lost” time and learn as much as possible, as fast as possible. So I have become a bit of an addict in experimenting, reading and writing, with a special focus on software design and architecture. That’s why I write these posts, to help me learn.</p><p>In my last posts, I’ve been writing about many of the concepts and principles that I’ve learned and a bit about how I reason about them. But I see these as just pieces of big a puzzle.</p><p>Today’s post is about how I fit all of these pieces together and, as it seems I should give it a name, I call it <strong>Explicit Architecture</strong>. Furthermore, these concepts have all “<em>passed their battle trials</em>” and are used in production code on highly demanding platforms. One is a SaaS e-com platform with thousands of web-shops worldwide, another one is a marketplace, live in 2 countries with a message bus that handles over 20 million messages per month.</p><ul><li>Fundamental blocks of the system</li><li>Tools</li><li>Connecting the tools and the Application Core<br>- Ports<br>- Primary or Driving Adapters<br>- Secondary or Driven Adapters<br>- Inversion of control</li><li>Application Core Organisation<br>- Application Layer<br>- Domain Layer<br> — Domain Services<br>— Domain Model</li><li>Components<br>- Decoupling the components<br> — Triggering logic in other components<br> — Getting data from other components<br> — — Data storage shared between components<br> — — Data storage segregated per component</li><li>Flow of control</li></ul><h3>Fundamental blocks of the system</h3><p>I start by recalling <a href="https://herbertograca.com/2017/08/24/ebi-architecture/"><strong>EBI</strong></a> and <a href="https://herbertograca.com/2017/09/14/ports-adapters-architecture/"><strong>Ports &amp; Adapters</strong></a> architectures. Both of them make an explicit separation of what code is internal to the application, what is external, and what is used for connecting internal and external code.</p><p>Furthermore, <a href="https://herbertograca.com/2017/09/14/ports-adapters-architecture/"><strong>Ports &amp; Adapters</strong></a> architecture explicitly identifies three fundamental blocks of code in a system:</p><ul><li>What makes it possible to run a <strong>user interface</strong>, whatever type of user interface it might be;</li><li>The system <strong>business logic</strong>, or <strong>application core</strong>, which is used by the user interface to actually make things happen;</li><li><strong>Infrastructure</strong> code, that connects our application core to tools like a database, a search engine or 3rd party APIs.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LI5lPrEFbAyKKPHf." /></figure><p>The application core is what we should really care about. It is the code that allows our code to do what it is supposed to do, it IS our application. It might use several user interfaces (progressive web app, mobile, CLI, API, …) but the code actually doing the work is the same and is located in the application core, it shouldn’t really matter what UI triggers it.</p><p>As you can imagine, the typical application flow goes from the code in the user interface, through the application core to the infrastructure code, back to the application core and finally deliver a response to the user interface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Q-59lmL9ETNbgaoL." /></figure><p>Far away from the most important code in our system, the application core, we have the tools that our application uses, for example, a database engine, a search engine, a Web server or a CLI console (although the last two are also delivery mechanisms).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JRqCLQkuPiGF5KhU." /></figure><p>While it might feel weird to put a CLI console in the same “bucket” as a database engine, and although they have different types of purposes, they are in fact tools used by the application. The key difference is that, while the CLI console and the web server are used to <strong>tell our application to do something</strong>, the database engine is <strong>told by our application to do something</strong>. This is a very relevant distinction, as it has strong implications on how we build the code that connects those tools with the application core.</p><h3>Connecting the tools and delivery mechanisms to the Application Core</h3><p>The code units that connect the tools to the application core are called adapters (<a href="https://herbertograca.com/2017/09/14/ports-adapters-architecture/">Ports &amp; Adapters Architecture</a>). The adapters are the ones that effectively implement the code that will allow the business logic to communicate with a specific tool and vice-versa.</p><p>The adapters that <strong>tell</strong> our application to do something are called <strong>Primary or Driving Adapters</strong> while the ones that are <strong>told</strong> by our application to do something are called <strong>Secondary or Driven Adapters</strong>.</p><h4>Ports</h4><p>These <em>Adapters</em>, however, are not randomly created. They are created to fit a very specific entry point to the Application Core, a <strong><em>Port</em></strong>. A port <strong>is nothing more than a specification</strong> of how the tool can use the application core, or how it is used by the Application Core. In most languages and in its most simple form, this specification, the Port, will be an Interface, but it might actually be composed of several Interfaces and DTOs.</p><p>It’s important to note that <strong>the Ports (Interfaces) belong inside the business logic</strong>, while the adapters belong outside. For this pattern to work as it should, it is of utmost importance that the Ports are created to fit the Application Core needs and not simply mimic the tools APIs.</p><h4>Primary or Driving Adapters</h4><p>The Primary or <strong>Driver Adapters wrap around a Port</strong> and use it to tell the Application Core what to do. <strong>They translate whatever comes from a delivery mechanism into a method call in the Application Core.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-qOBRSYGC7cufZ3Z." /></figure><p>In other words, our Driving Adapters are Controllers or Console Commands who are injected in their constructor with some object whose class implements the interface (Port) that the controller or console command requires.</p><p>In a more concrete example, a Port can be a Service interface or a Repository interface that a controller requires. The concrete implementation of the Service, Repository or Query is then injected and used in the Controller.</p><p>Alternatively, a Port can be a Command Bus or Query Bus interface. In this case, a concrete implementation of the Command or Query Bus is injected into the Controller, who then constructs a Command or Query and passes it to the relevant Bus.</p><h4>Secondary or Driven Adapters</h4><p>Unlike the Driver Adapters, who wrap around a port, <strong>the Driven Adapters implement a Port</strong>, an interface, and are then injected into the Application Core, wherever the port is required (type-hinted).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*C0vYQ4pup-UXLDEA." /></figure><p>For example, let’s suppose that we have a naive application which needs to persist data. So we create a persistence interface that meets its needs, with a method to <em>save</em> an array of data and a method to <em>delete</em> a line in a table by its ID. From then on, wherever our application needs to save or delete data we will require in its constructor an object that implements the persistence interface that we defined.</p><p>Now we create an adapter specific to MySQL which will implement that interface. It will have the methods to save an array and delete a line in a table, and we will inject it wherever the persistence interface is required.</p><p>If at some point we decide to change the database vendor, let’s say to PostgreSQL or MongoDB, we just need to create a new adapter that implements the persistence interface and is specific to PostgreSQL, and inject the new adapter instead of the old one.</p><h3>Inversion of control</h3><p>A characteristic to note about this pattern is that the adapters depend on a specific tool and a specific port (by implementing an interface). But our business logic only depends on the port (interface), which is designed to fit the business logic needs, so it doesn’t depend on a specific adapter or tool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w6fykyVD04tqRYz2." /></figure><p>This means the direction of dependencies is towards the centre, it’s the <strong>inversion of control principle at the architectural level</strong>.</p><p>Although, again, <strong>it is of utmost importance that the Ports are created to fit the Application Core needs and not simply mimic the tools APIs.</strong></p><h3>Application Core organisation</h3><p>The <a href="https://herbertograca.com/2017/09/21/onion-architecture/">Onion Architecture</a> picks up the DDD layers and incorporates them into the <a href="https://herbertograca.com/2017/09/14/ports-adapters-architecture/">Ports &amp; Adapters Architecture</a>. Those layers are intended to bring some organisation to the business logic, the interior of the Ports &amp; Adapters “hexagon”, and just like in Ports &amp; Adapters, the dependencies direction is towards the centre.</p><h4>Application Layer</h4><p>The use cases are the processes that can be triggered in our Application Core by one or several User Interfaces in our application. For example, in a CMS we could have the actual application UI used by the common users, another independent UI for the CMS administrators, another CLI UI, and a web API. These UIs (applications) could trigger use cases that can be specific to one of them or reused by several of them.</p><p>The use cases are defined in the Application Layer, the first layer provided by DDD and used by the Onion Architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ik1SZ997_fZ-vi8o." /></figure><p>This layer contains Application Services (and their interfaces) as first class citizens, but it also contains the Ports &amp; Adapters interfaces (ports) which include ORM interfaces, search engines interfaces, messaging interfaces and so on. In the case where we are using a Command Bus and/or a Query Bus, this layer is where the respective Handlers for the Commands and Queries belong.</p><p>The Application Services and/or Command Handlers contain the logic to unfold a use case, a business process. Typically, their role is to:</p><ol><li>use a repository to find one or several entities;</li><li>tell those entities to do some domain logic;</li><li>and use the repository to persist the entities again, effectively saving the data changes.</li></ol><p>The Command Handlers can be used in two different ways:</p><ol><li>They can contain the actual logic to perform the use case;</li><li>They can be used as mere wiring pieces in our architecture, receiving a Command and simply triggering logic that exists in an Application Service.</li></ol><p>Which approach to use depends on the context, for example:</p><ul><li>Do we already have the Application Services in place and are now adding a Command Bus?</li><li>Does the Command Bus allow specifying any class/method as a handler, or do they need to extend or implement existing classes or interfaces?</li></ul><p>This layer also contains the triggering of <strong>Application Events</strong>, which represent some outcome of a use case. These events trigger logic that is a side effect of a use case, like sending emails, notifying a 3rd party API, sending a push notification, or even starting another use case that belongs to a different component of the application.</p><h4>Domain Layer</h4><p>Further inwards, we have the Domain Layer. The objects in this layer contain the data and the logic to manipulate that data, that is specific to the Domain itself and it’s independent of the business processes that trigger that logic, they are independent and completely unaware of the Application Layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qg893h39U5L495fK." /></figure><h4>Domain Services</h4><p>As I mentioned above, the role of an Application Service is to:</p><ol><li>use a repository to find one or several entities;</li><li>tell those entities to do some domain logic;</li><li>and use the repository to persist the entities again, effectively saving the data changes.</li></ol><p>However, sometimes we encounter some domain logic that involves different entities, of the same type or not, and we feel that that domain logic does not belong in the entities themselves, we feel that that logic is not their direct responsibility.</p><p>So our first reaction might be to place that logic outside the entities, in an Application Service. However, this means that that domain logic will not be reusable in other use cases: domain logic should stay out of the application layer!</p><p>The solution is to create a Domain Service, which has the role of receiving a set of entities and performing some business logic on them. A Domain Service belongs to the Domain Layer, and therefore it knows nothing about the classes in the Application Layer, like the Application Services or the Repositories. In the other hand, it can use other Domain Services and, of course, the Domain Model objects.</p><h4>Domain Model</h4><p>In the very centre, depending on nothing outside it, is the Domain Model, which contains the business objects that represent something in the domain. Examples of these objects are, first of all, Entities but also Value Objects, Enums and any objects used in the Domain Model.</p><p>The Domain Model is also where Domain Events “live”. These events are triggered when a specific set of data changes and they carry those changes with them. In other words, when an entity changes, a Domain Event is triggered and it carries the changed properties new values. These events are perfect, for example, to be used in Event Sourcing.</p><h3>Components</h3><p>So far we have been segregating the code based on layers, but that is the fine-grained code segregation. The coarse-grained segregation of code is at least as important and it’s about segregating the code according to sub-domains and <a href="http://ddd.fed.wiki.org/view/welcome-visitors/view/domain-driven-design/view/bounded-context"><strong><em>bounded contexts</em></strong></a>, following Robert C. Martin ideas expressed in <a href="https://8thlight.com/blog/uncle-bob/2011/09/30/Screaming-Architecture.html"><strong><em>screaming architecture</em></strong></a>. This is often referred to as Packaging by Component as opposed to Packaging by Layer, and it’s quite well explained by Simon Brown in <a href="http://www.codingthearchitecture.com/2015/03/08/package_by_component_and_architecturally_aligned_testing.html">this blog post</a>. These sections of code are cross-cutting to the layers previously described, they are the <a href="https://herbertograca.com/2017/07/05/software-architecture-premises/"><strong><em>components</em></strong></a> of our application. Examples of components can be Authentication, Authorization, Billing, Users or Accounts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TyOegqec1897L9Fh." /></figure><h3>Decoupling the components</h3><p>Just like the fine-grained code units (classes, interfaces, traits, mixins, …), also the coarsely grained code-units (components) benefit from low coupling and high cohesion.</p><p>To decouple classes we make use of Dependency Injection, by injecting dependencies into a class as opposed to instantiating them inside the class, and Dependency Inversion, by making the class depend on abstractions (interfaces and/or abstract classes) instead of concrete classes. This means that the depending class has no knowledge about the concrete class that it is going to use, it has no reference to the fully qualified class name of the classes that it depends on.</p><p>In the same way, having completely decoupled components means that a component has no direct knowledge of any another component. In other words, it has no reference to any fine-grained code unit from another component, not even interfaces! This means that Dependency Injection and Dependency Inversion are not enough to decouple components, we will need some sort of architectural constructs. We might need events, a shared kernel, eventual consistency, and even a discovery service!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FMAWaWbBSgRL57vf." /></figure><h4>Triggering logic in other components</h4><p>When one of our components (component B) needs to do something whenever something else happens in another component (component A), we can not simply make a direct call from component A to a class/method in component B because A would then be coupled to B.</p><p>However we can make A use an event dispatcher to dispatch an application event that will be delivered to any component listening to it, including B, and the event listener in B will trigger the desired action. This means that component A will depend on an event dispatcher, but it will be decoupled from B.</p><p>Nevertheless, if the event itself “lives” in A this means that B knows about the existence of A, it is coupled to A. To remove this dependency, we can create a library with a set of application core functionality that will be shared among all components, the <a href="http://ddd.fed.wiki.org/view/welcome-visitors/view/domain-driven-design/view/shared-kernel">Shared Kernel</a>. This means that the components will both depend on the Shared Kernel but they will be decoupled from each other. The Shared Kernel will contain functionality like application and domain events, but it can also contain Specification objects, and whatever makes sense to share, keeping in mind that it should be as minimal as possible because any changes to the Shared Kernel will affect all components of the application. Furthermore, if we have a polyglot system, let’s say a micro-services ecosystem where they are written in different languages, the Shared Kernel needs to be language agnostic so that it can be understood by all components, whatever the language they have been written in. For example, instead of the Shared Kernel containing an Event class, it will contain the event description (ie. name, properties, maybe even methods although these would be more useful in a Specification object) in an agnostic language like JSON, so that all components/micro-services can interpret it and maybe even auto-generate their own concrete implementations.</p><p>This approach works both in monolithic applications and distributed applications like micro-services ecosystems. However, when the events can only be delivered asynchronously, for contexts where triggering logic in other components needs to be done immediately this approach will not suffice! Component A will need to make a direct HTTP call to component B. In this case, to have the components decoupled, we will need a discovery service to which A will ask where it should send the request to trigger the desired action, or alternatively make the request to the discovery service who can proxy it to the relevant service and eventually return a response back to the requester. This approach will couple the components to the discovery service but will keep them decoupled from each other.</p><h4>Getting data from other components</h4><p>The way I see it, a component is not allowed to change data that it does not “own”, but it is fine for it to query and use any data.</p><p>When a component needs to use data that belongs to another component, let’s say a billing component needs to use the client name which belongs to the accounts component, the billing component will contain a query object that will query the data storage for that data. This simply means that the billing component can know about any dataset, but it must use the data that it does not “own” as read-only, by the means of queries.</p><p>Data storage segregated per component</p><p>In this case, the same pattern applies, but we have more complexity at the data storage level. Having components with their own data storage means each data storage contains:</p><ul><li>A set of data that it owns and is the only one allowed to change, making it the single source of truth;</li><li>A set of data that is a copy of other components data, which it can not change on its own, but is needed for the component functionality, and it needs to be updated whenever it changes in the owner component.</li></ul><p>Each component will create a local copy of the data it needs from other components, to be used when needed. When the data changes in the component that owns it, that owner component will trigger a domain event carrying the data changes. The components holding a copy of that data will be listening to that domain event and will update their local copy accordingly.</p><h3>Flow of control</h3><p>As I said above, the flow of control goes, of course, from the user into the Application Core, over to the infrastructure tools, back to the Application Core and finally back to the user. But how exactly do classes fit together? Which ones depend on which ones? How do we compose them?</p><p>Following Uncle Bob, in his article about Clean Architecture, I will try to explain the flow of control with UMLish diagrams…</p><h3>Without a Command/Query Bus</h3><p>In the case we do not use a command bus, the Controllers will depend either on an Application Service or on a Query object.</p><p>[<strong>EDIT — 2017–11–18</strong>] I completely missed the DTO I use to return data from the query, so I added it now. Tkx to <a href="https://www.reddit.com/user/MorphineAdministered">MorphineAdministered</a> who <a href="https://www.reddit.com/r/PHP/comments/7dcz8k/ddd_hexagonal_onion_clean_cqrs_how_i_put_it_all/dpy6va4/">pointed it out</a> for me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/0*mNtB0UxhDVgQ68CI." /></figure><p>In the diagram above we use an interface for the Application Service, although we might argue that it is not really needed since the Application Service is part of our application code and we will not want to swap it for another implementation, although we might refactor it entirely.</p><p>The Query object will contain an optimized query that will simply return some raw data to be shown to the user. That data will be returned in a DTO which will be injected into a ViewModel. ThisViewModel may have some view logic in it, and it will be used to populate a View.</p><p>The Application Service, on the other hand, will contain the use case logic, the logic we will trigger when we want to do something in the system, as opposed to simply view some data. The Application Services depend on Repositories which will return the Entity(ies) that contain the logic which needs to be triggered. It might also depend on a Domain Service to coordinate a domain process in several entities, but that is hardly ever the case.</p><p>After unfolding the use case, the Application Service might want to notify the whole system that that use case has happened, in which case it will also depend on an event dispatcher to trigger the event.</p><p>It is interesting to note that we place interfaces both on the persistence engine and on the repositories. Although it might seem redundant, they serve different purposes:</p><ul><li>The persistence interface is an abstraction layer over the ORM so we can swap the ORM being used with no changes to the Application Core.</li><li>The repository interface is an abstraction on the persistence engine itself. Let’s say we want to switch from MySQL to MongoDB. The persistence interface can be the same, and, if we want to continue using the same ORM, even the persistence adapter will stay the same. However, the query language is completely different, so we can create new repositories which use the same persistence mechanism, implement the same repository interfaces but builds the queries using the MongoDB query language instead of SQL.</li></ul><h3>With a Command/Query Bus</h3><p>In the case that our application uses a Command/Query Bus, the diagram stays pretty much the same, with the exception that the controller now depends on the Bus and on a command or a Query. It will instantiate the Command or the Query, and pass it along to the Bus who will find the appropriate handler to receive and handle the command.</p><p>In the diagram below, the Command Handler then uses an Application Service. However, that is not always needed, in fact in most of the cases the handler will contain all the logic of the use case. We only need to extract logic from the handler into a separated Application Service if we need to reuse that same logic in another handler.</p><p>[<strong>EDIT — 2017–11–18</strong>] I completely missed the DTO I use to return data from the query, so I added it now. Tkx to <a href="https://www.reddit.com/user/MorphineAdministered">MorphineAdministered</a> who <a href="https://www.reddit.com/r/PHP/comments/7dcz8k/ddd_hexagonal_onion_clean_cqrs_how_i_put_it_all/dpy6va4/">pointed it out</a> for me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/0*lgiBmjm2JzvdB-2-." /></figure><p>You might have noticed that there is no dependency between the Bus and the Command, the Query nor the Handlers. This is because they should, in fact, be unaware of each other in order to provide for good decoupling. The way the Bus will know what Handler should handle what Command, or Query, should be set up with mere configuration.</p><p>As you can see, in both cases all the arrows, the dependencies, that cross the border of the application core, they point inwards. As explained before, this a fundamental rule of Ports &amp; Adapters Architecture, Onion Architecture and Clean Architecture.</p><h3>Conclusion</h3><p>The goal, as always, is to have a code-base that is loosely coupled and high cohesive, so that changes are easy, fast and safe to make.</p><blockquote>Plans are worthless, but planning is everything.<br>Eisenhower</blockquote><p>This info-graphic is a concept map. Knowing and understanding all of these concepts will help us plan for a healthy architecture, a healthy application.</p><p>Nevertheless:</p><blockquote>The map is not the territory.<br>Alfred Korzybski</blockquote><p>Meaning that <strong>these are just guidelines!</strong> <strong>The application is the territory, the reality, the concrete use case where we need to apply our knowledge, and that is what will define what the actual architecture will look like!</strong></p><p><strong>We need to understand all these patterns, but we also always need to think and understand exactly what our application needs, how far should we go for the sake decoupling and cohesiveness.</strong> This decision can depend on plenty of factors, starting with the project functional requirements, but can also include factors like the time-frame to build the application, the lifespan of the application, the experience of the development team, and so on.</p><p>This is it, this is how I make sense of it all. This is how I rationalize it in my head.</p><p>However, how do we make all this explicit in the code base? That’s the subject of my next post about how I reflect the architecture and domain, in the code.</p><p>Last but not least, thanks to my colleague <a href="https://www.linkedin.com/in/francescomastrogiacomo/">Francesco Mastrogiacomo</a>, for helping me make my infographic look nice. :)</p><p><em>Originally published at </em><a href="https://herbertograca.com/2017/11/16/explicit-architecture-01-ddd-hexagonal-onion-clean-cqrs-how-i-put-it-all-together/"><em>herbertograca.com</em></a><em> on November 16, 2017.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2590c0aa7f6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-software-architecture-chronicles/ddd-hexagonal-onion-clean-cqrs-how-i-put-it-all-together-f2590c0aa7f6">DDD, Hexagonal, Onion, Clean, CQRS, … How I put it all together</a> was originally published in <a href="https://medium.com/the-software-architecture-chronicles">The Software Architecture Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Service Oriented Architecture (SOA)]]></title>
            <link>https://medium.com/the-software-architecture-chronicles/service-oriented-architecture-soa-72fdb7dbebfb?source=rss-6cdf53b3244------2</link>
            <guid isPermaLink="false">https://medium.com/p/72fdb7dbebfb</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[soa]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Herberto Graça]]></dc:creator>
            <pubDate>Thu, 09 Nov 2017 14:30:18 GMT</pubDate>
            <atom:updated>2017-12-25T08:01:01.394Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is part of </em><a href="https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/"><em>The Software Architecture Chronicles</em></a><em>, a </em><a href="https://herbertograca.com/category/development/series/software-architecture/"><em>series of posts about Software Architecture</em></a><em>. In them, I write about what I’ve learned on Software Architecture, how I think of it, and how I use that knowledge. The contents of this post might make more sense if you read the previous posts in this series.</em></p><p>The SOA Style has been around since the late 1980s and has its origins in ideas introduced by CORBA, DCOM, DCE and others. Much has been said about SOA, and there are a few different implementation patterns but, in essence, SOA focuses on only a few concepts and doesn’t give any prescription on how to implement them:</p><ul><li>Composability of user-facing applications;</li><li>Reusable Business Services;</li><li>Technology stack independent;</li><li>Autonomy (independent evolution, scalability &amp; deployability).</li></ul><p>SOA is a set of architectural principles independent of any technology or product, just like polymorphism and encapsulation are.</p><p>In this post I am going to address the following patterns related to SOA:</p><ul><li>CORBA — Common Object Request Broker Architecture</li><li>Web Services</li><li>Message Queue</li><li>Enterprise Service Bus (ESB)</li><li>Microservices</li></ul><p><strong>CORBA — Common Object Request Broker Architecture</strong></p><p>During the 1980s, derived from the growing usage of enterprise networks and the client/server architecture, there was an increasing need for a common way of getting applications that were built using different technologies and were running on different computers with a different OS, to communicate. CORBA was developed to fulfil those needs. It is a standard, for Distributed Computing, that developed during the 1980s and reached its first mature version 1991.</p><p>The CORBA standard was implemented by several vendors and aimed at providing:</p><ul><li>Platform neutral Remote Procedure Call;</li><li>Transactions (also remote transactions!!);</li><li>Security;</li><li>Events;</li><li>Programming language independence;</li><li>OS independence;</li><li>Hardware independence;</li><li>Isolation from data-transfer/communication details;</li><li>Data typing through an Interface Definition Language (IDL).</li></ul><p>At the moment CORBA is still used for heterogeneous computing, for example, <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/idl/corba.html">it is still part of JAVA EE</a>, although <a href="https://www.infoq.com/news/2016/06/corba-removed-java-9">it will be packaged as a separate module from JAVA 9 onwards</a>.</p><p>It’s important to note, though, that <strong>I don’t consider CORBA to be an SOA pattern</strong> (although I do consider both CORBA and SOA patterns to be under the umbrella of distributed computing). I chose to include it here because I feel it was <strong>its shortcomings that led to the SOA</strong> movement.</p><h3>How it works</h3><p>First, we need to acquire an Object Request Broker (ORB), which conforms to the CORBA specification, it is supplied by a vendor and it uses language mappers to generate stubs and skeletons in the client code languages. Using that ORB and the interface definitions defined using an IDL (similar to a WSDL), in the client, we generate <strong>stub</strong> classes of the real classes that can be called remotely and, in the server, we generate <strong>skeleton</strong> classes that can handle incoming requests and make the call to the real target object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/0*uVhnfdpkmBPpjhmD." /></figure><ol><li>The caller calls a local procedure implemented by the stub;</li><li>The stub validates the call and creates a request message that passes to the ORB;</li><li>The client ORB sends the message over the network to the server and blocks the current execution thread;</li><li>The server ORB receives the request message and instantiates the skeleton;</li><li>The skeleton executes the procedure on the called object;</li><li>The called object performs a computation and returns the result;</li><li>The skeleton packs the output arguments into a response message and passes it to the ORB;</li><li>The ORB sends the message over the network back to the client;</li><li>The client ORB receives the response message, unpacks it and delivers it to the stub;</li><li>The stub passes output arguments to the caller, releases execution thread and the caller continues in execution.</li></ol><h3>Pros</h3><ul><li>Tech stack independence (except for the ORB implementation);</li><li>Isolation from data-transfer/communication details.</li></ul><h3>Cons</h3><ul><li><strong>Location transparency: </strong>The client code does not know if a call is local or remote. This sounds like a good thing, however, the latency and type of failures are quite different, depending if it’s a local or remote call. Not knowing what type of call it is, makes it impossible for applications to choose an appropriate strategy to handle the method call, and to end up making remote calls inside a loop, and therefore considerably slowing down the whole system.</li><li><strong>Complex, redundant and ambiguous specification: </strong>It was created as a mash-up of several existing vendor versions, so (at that time) it was ambiguous and redundant, making it difficult to implement.</li><li><strong>Blocked communication pipes: </strong>It used specific protocols over TCP/IP and specific ports, or even random ports. But enterprise security rules and firewalls would often only allow for HTTP communication over port 80, effectively blocking CORBA communication.</li></ul><h3>Web Services</h3><p>Although CORBA still has its use cases nowadays, we learned that <strong>we needed to reduce the remote communications</strong> to make the system more performant, <strong>we needed a reliable communication pipe</strong>, and <strong>we needed a simpler communication specification</strong>.</p><p>Thus, during the late 1990s, web services started to spawn with the goals of solving the problems mentioned above:</p><p><strong>We needed a reliable communication pipe, so:</strong></p><ul><li><strong>HTTP through port 80</strong> is the default communication pipe;</li><li>An <strong>agnostic communication language</strong> is used (like <strong>XML</strong> or <strong>JSON</strong>);</li></ul><p><strong>We needed to reduce the remote communications, so:</strong></p><ul><li>We have <strong>explicit remote communications</strong> so that we know exactly when we are making a remote call;</li><li>We have <strong>coarse-grained remote calls</strong>, ie. instead of calling remote objects often, we call a remote service less often;</li></ul><p><strong>We needed a simpler communication specification, so:</strong></p><ul><li><strong>SOAP</strong> had its first draft in 1998 and reached a W3C recommendation in 2003, making it effectively a standard. It embodied some of the ideas of CORBA, like a layer to handle the communication and a “document” defining the interfaces using a <em>Web Services Description Language</em> (WSDL);</li><li><strong>REST</strong> was defined in 2000, by Roy Fielding in his PhD dissertation “<em>Architectural Styles and the Design of Network-based Software Architectures</em>“, and it is a much simpler specification than SOAP which made it quickly gain higher adoption than the slightly older SOAP specification.</li><li><strong>GraphQL</strong> was developed by Facebook in 2012 and released to the public in 2015. It is an API query language that allows the client to specify exactly what data the server should send back, as opposed to an ad-hoc payload, avoiding both over-fetching and under-fetching data.</li></ul><blockquote><em>[Web] Services can be published, discovered and used in a technology neutral, standard form.</em></blockquote><blockquote><em>Microsoft 2004, </em><a href="https://msdn.microsoft.com/en-us/library/aa480021.aspx"><em>Understanding Service-Oriented Architecture</em></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/0*PU8QYrwYS72PswVr." /></figure><p>Through Web services, SOA makes a paradigm shift from one of remotely invoking methods on objects (CORBA), to one of passing messages between services.</p><p>We need to understand, though, that under the umbrella of SOA, a web service is not just a general purpose API, which simply provides CRUD access to its database through HTTP. While that implementation might be useful in some cases, <em>it requires that users understand the underlying model and comply with the business rules to ensure that your data integrity is protected</em>. SOA implies that web services are designed as bounded contexts for business sub-domains, abstracting the implementation from the conceptual service they provide.</p><blockquote><em>SOA is not just an architecture of services seen from a technology perspective, but the policies, practices, and frameworks by which we ensure the right services are provided and consumed.</em></blockquote><blockquote><em>Microsoft 2004, </em><a href="https://msdn.microsoft.com/en-us/library/aa480021.aspx"><em>Understanding Service-Oriented Architecture</em></a></blockquote><h3>Pros</h3><ul><li>Independent tech stack, deployment and scalability of services;</li><li><strong>Common, simple and reliable communication channel (text through HTTP, port 80);</strong></li><li><strong>Optimised communication;</strong></li><li><strong>Stable communication specification;</strong></li><li><strong>Insulation of Domain contexts.</strong></li></ul><h3>Cons</h3><ul><li><strong>Difficult integration</strong> of different web services, due to different communication languages, ie. two web services that use different JSON representations of the same concept;</li><li>Synchronous communication can <strong>overload</strong> the systems.</li></ul><h3>Message Queue</h3><p>The core idea is to have several applications communicating asynchronously between them, using agnostic messages. The Message Queue provides for improved scalability and higher decoupling between the applications as they don’t need to know where the other applications are located, how many and not even who they are. Despite this, they all need to use the same communication language, ie a predetermined text format to represent data.</p><p>A Message Queue uses a message broker software (ie. RabbitMQ, Beanstalkd, Kafka, …) as an infrastructure artefact and can be set up in different ways to implement the communication between applications:</p><p><strong>Request/Reply</strong></p><ul><li>A client sends a message to the Message Queue, including a “<em>conversation</em>” reference. The message is delivered to a specific node, which will reply with another message, to the original sender, containing the same <em>conversation </em>reference so that the receiver knows to what <em>conversation</em> the message refers to and it can continue the process. It’s very useful for medium and long-running business processes (<strong><em>sagas</em></strong>).</li></ul><p><strong>Publish/Subscribe</strong></p><ul><li><strong>List-Based</strong><br> It maintains lists of published topics and subscribers to those topics. As it receives messages for a topic it puts the message into the corresponding topic list. Matching the message to a topic can be done by the message type or with a more complex set of predetermined criteria that can include the message content itself.</li><li><strong>Broadcast-Based</strong><br> As it receives messages, it broadcasts them to all the nodes that are listening to the queue. Each listening node is responsible for filtering and processing only the messages that it is interested in.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/0*h5UNv4rwu7E4n6HS." /></figure><p>All these patterns can be set up in a <strong><em>pull</em></strong> (aka <em>polling)</em> or <strong><em>push</em></strong> approach<em>:</em></p><ul><li>In a <strong>pull scenario</strong>, the client will ask the queue for a message at every <em>X</em> amount of time. This has the benefit that <strong>the client is able to control its load</strong> but also might have the downside of <strong>introducing latency</strong>, ie. when there are messages on the queue and the client is not handling messages but just waiting for the moment to poll for a new message;</li><li>In a <strong>push scenario</strong>, the queue will push messages to clients as soon as a message is received. The advantage here is that there is <strong>no latency</strong>, but the <strong>clients can not self-manage their load</strong>.</li></ul><h3>Pros</h3><ul><li>Independent tech stack, deployment and scalability of services;</li><li>Common, simple and reliable communication channel (text through HTTP, port 80);</li><li>Optimised communication;</li><li>Stable communication specification;</li><li>Insulation of Domain contexts;</li><li><strong>Easy attachment and detachment of services;</strong></li><li><strong>Asynchronous communication helps manage the system’s load.</strong></li></ul><h3>Cons</h3><ul><li><strong>Difficult integration</strong> of different web services, due to different communication languages, ie. two web services that use different JSON representations of the same concept;</li></ul><h3>Enterprise Service Bus (ESB)</h3><p>During the 1990s, at the same time that the Web Services were evolving, the Enterprise Service Bus was already making use of them (maybe some implementations even used CORBA initially?).</p><p>The ESB came to life in a context where the companies had their stand-alone applications, like one application for finance, another one for human resources, another one for stock management, etc., and they needed to make those applications communicate with each other, they needed to integrate them. But those applications had not been thought with integration in mind, there was no common language format for applications to communicate (as there isn’t one today). So the reasonable solution was for the application vendors to create endpoints to send and receive data in a specific format. The client companies would then have to integrate the applications by establishing a communication pipe and translate the messages from one application language to another.</p><p>The Message Queue could contribute to solving this problem, but it still could not solve the problem of applications having different language formats. Nevertheless, it was a small step to change the Message Queue from a dumb communication channel into a middleware that handles both delivering messages and their transformation into the language/format expected by the receiver. The ESB was a natural evolution of the simpler Message Queue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/290/0*WL4RJqQ39R37u21o." /></figure><p>In this architecture type, we have composite applications, typically user-facing, that contact web services to perform some operation. This web services, in turn, can also contact other web services and in the end, they may return some data back to the composite application. However, both the composite application and the back-end services have no knowledge of the details of each other, namely their location or communication protocols. What they do know is what service they need and the location of the Service Bus.</p><p>So the client application (be it a service or a composite application) sends its request to the Service Bus, which in turn <strong>transforms</strong> the message into the format expected by the destination and <strong>routes</strong> the request to the destination. It’s important to note that <strong>all</strong> communication goes through the ESB, which means that if the ESB goes down, all communications go down and all systems are rendered inoperative. In the end, an ESB works like a middleware where a lot of things happen, rendering it a highly complex artefact.</p><p>This was, of course, a very basic explanation of what an ESB architecture is. Furthermore, although the ESB is the main artefact in this architecture, there can be other artefacts involved, like Domain Brokers, Data Services, Process Orchestration Services or Rules Engines. This same architecture pattern can also be set up in a federated design, where the system is segregated in business domains, each with its own ESB setup and all of those setups connected between them. This helps to improve performance and mitigate the problem of having a single point of failure, ie. if one ESB fails, only its business domain will be affected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JOfM_nv2SQryQNu4." /></figure><p>The primary duties of an ESB are:</p><ul><li>Monitor and control <strong>routing</strong> of message exchange between services;</li><li>Resolve <strong>translation of messages</strong> between communicating service components;</li><li>Control <strong>deployment</strong> and <strong>versioning</strong> of services;</li><li>Marshal use of <strong>redundant services</strong>;</li><li>Cater for <strong>commodity services</strong> like event handling, data transformation and mapping, message and event queuing and sequencing, security or exception handling, protocol conversion and enforcing proper quality of communication service;</li></ul><blockquote><em>When building communication structures between different processes, we’ve seen many products and approaches that stress putting significant smarts into the communication mechanism itself. A good example of this is the Enterprise Service Bus (ESB), where ESB products often include sophisticated facilities for message routing, choreography, transformation, and applying business rules.</em></blockquote><blockquote><em>Martin Fowler 2014, </em><a href="https://martinfowler.com/articles/microservices.html"><em>Microservices</em></a></blockquote><p>This architecture pattern has advantages but I find it especially useful if we don’t “<em>own</em>” the web services and therefore need a <strong>middleware</strong> to translate the messages between them, orchestrate business processes involving several web services, etc..</p><p>It’s also good to keep in mind that the ESB implementations have evolved and today we can even use a simple drag &amp; drop UI to configure an ESB, for most use cases.</p><h3>Pros</h3><ul><li>Independent tech stack, deployment and scalability of services;</li><li>Common, simple and reliable communication channel (text through HTTP, port 80);</li><li>Optimised communication;</li><li>Stable communication specification;</li><li>Insulation of Domain contexts;</li><li>Easy attachment and detachment of services;</li><li>Asynchronous communication helps manage the system’s load;</li><li><strong>Single point for versioning and translation management.</strong></li></ul><h3>Cons</h3><ul><li>Slower communication speed, especially for those already compatible services;</li><li><strong>Centralised logic</strong>:</li><li>Single point of failure that can bring down all communications in the Enterprise;</li><li>High configuration and maintenance complexity;</li><li>In time the ESB can end up containing business rules;</li><li>Due to its complexity it will eventually need a team just to manage it;</li><li>Services become highly dependent of the ESB.</li></ul><h3>Microservices</h3><p>The Microservices Architecture has its foundations on the SOA concepts, and shares the same global goals as the ESB: Creating one global enterprise application from several more specific business domain applications.</p><p>The key difference is that the ESB was born into a context of <strong>stand-alone applications that needed to be integrated</strong> in order to achieve an enterprise-wide distributed application, while the Microservices Architecture was born into a context of fast paced and ever changing <strong>businesses who (mostly) create their own cloud applications</strong> from scratch.</p><p>In other words, the starting point is different. In the case of the ESB, we start with <strong>existing applications which we do not “own”</strong>, and therefore we can not change them. But with the Microservices, we have <strong>full control over the application(s)</strong> (which doesn’t mean there can’t be any 3rd party web services involved in the system).</p><p>The way microservices are built/designed prevents a high need for integration. Microservices should be specific to a business concept, to a bounded context, they should keep their own state so that they don’t depend directly on other microservices and therefore need less integration. In other words, the low coupling and high cohesion provided by microservices have the nice side effect of reducing the need for integration.</p><blockquote><em>[Microservices are] Small autonomous services that work together, modelled around a business domain.</em></blockquote><blockquote><em>Sam Newman 2015, </em><a href="https://youtu.be/PFQnNFe27kU?t=1m50s"><em>Principles Of Microservices</em></a></blockquote><p>Since the biggest drawback of the ESB Architecture was the very complex and central application of which all other applications would depend on, the Microservices Architecture addresses that problem by removing it almost completely.</p><p>There are still elements that are cross-cutting to the whole microservices ecosystem, but they don’t contain so many responsibilities as the ESB would contain. For example, there still is a Message Queue used for asynchronous communication between microservices, but it is simply a message pipe with no other responsibilities. Another example is the microservice ecosystem gateway through which all communications with the outside are made.</p><p>Sam Newman, the author of <a href="https://www.amazon.co.uk/d/Books/Building-Microservices-Sam-Newman/1491950358"><em>Building Microservices</em></a>, identifies 8 principles of a Microservices Architecture:</p><ul><li><strong>Services are modelled around business domains<br> </strong>Because it can give us stable interfaces around a business concept, very cohesive and decoupled units of code and clearly identified bounded contexts;</li><li><strong>Culture of automation<br> </strong>Because we will have a lot more moving parts and deployable units;</li><li><strong>Hide implementation details</strong><br> To allow one service to evolve independently of another;</li><li><strong>Decentralise all the things<br> </strong>Decentralise the decision making power and the architectural concepts, giving autonomy to the teams so that the organisation transforms itself into a Complex Adaptative System who can quickly adapt to change;</li><li><strong>Deploy independently<br> </strong>So that we can deploy a new version of a service without the need to change anything else;</li><li><strong>Consumer first<br> </strong>A service should be easy to consume, easy to be used by other services;</li><li><strong>Isolate</strong> failure<br> So that even if one service fails, the others continue to operate, giving the overall system a high resilience to failure;</li><li><strong>Highly observable<br> </strong>Due to the system high number of parts, it is more difficult to understand everything that is going on, so we need sophisticated monitoring tools that allow us to know what is going on in every corner of the system and understand any chain reactions.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/0*v1AO71FdQI51qPxg." /></figure><blockquote><em>The microservice community favours an alternative approach: </em><strong><em>smart endpoints and dumb pipes</em></strong><em>. Applications built from microservices aim to be as decoupled and as cohesive as possible — they own their own domain logic and act more as filters in the classical Unix sense — receiving a request, applying logic as appropriate and producing a response. These are choreographed using simple RESTish protocols rather than complex protocols such as WS-Choreography or BPEL or orchestration by a central tool.</em></blockquote><blockquote><em>Martin Fowler 2014, </em><a href="https://martinfowler.com/articles/microservices.html"><em>Microservices</em></a></blockquote><h3>Pros</h3><ul><li>Independent tech stack, deployment and scalability of services;</li><li>Common, simple and reliable communication channel (text through HTTP, port 80);</li><li>Optimised communication;</li><li>Stable communication specification;</li><li>Insulation of Domain contexts;</li><li>Easy attachment and detachment of services;</li><li>Asynchronous communication helps manage the system’s load;</li><li><strong>Synchronous communication helps manage the system’s performance;</strong></li><li><strong>Truly independent and autonomous services;</strong></li><li><strong>No business logic outside of the services;</strong></li><li><strong>Potential to transform the organisation into a <em>Complex Adaptative System</em>, composed of several small autonomous parts/teams who can quickly adapt to business changes.</strong></li></ul><h3>Cons</h3><ul><li><strong>High operational complexity</strong>:</li><li>Needs high investment in a strong DevOps culture;</li><li>Using a multitude of technologies and libraries can get out of hand;</li><li>Input and output API changes must be managed carefully because there will be software relying on those interfaces;</li><li>The usage of eventual consistency has significant implications that must be addressed while developing the application, all the way from the back-end to the UX layers;</li><li>Testing becomes more complex as interface changes can have unpredictable consequences in other services.</li></ul><h3>Anti-Pattern: Ravioli Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/0*mnsFCWINeg8zJfvl." /></figure><p><em>Ravioli Architecture</em> is the name commonly used to refer to the <strong>anti-pattern for Microservices Architecture</strong>. It happens when we end up creating an ecosystem of micro-services where they are too many, too small and don’t represent domain concepts on their own.</p><h3>Conclusion</h3><p>SOA has evolved a lot in the last decades and, derived from both the insufficiencies of the implemented solutions and the technological advances, we have reached the Microservices Architecture.</p><p>The idea behind this whole evolution has always been the usual strategy used to solve complex problems: break the problem into smaller, solvable pieces.</p><p>Solving the code complexity can also be done in the same way when we have a monolith, breaking it up into decoupled domain components (bounded contexts). But, as the teams and the codebase grows, so does the need for independent evolution, scalability &amp; deployability. SOA provides the tools for this independence, forcing stricter boundaries between bounded contexts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/0*PJgFHhAz23ENJ2SP." /></figure><p>Once more, it’s about low coupling and high cohesion, this time at an even coarser granularity than before. It’s also, again, fundamental to analyse our needs with pragmatism: use SOA only when we really need to, because it brings a lot of complexity to the mix, and <strong>if we do really need to use SOA</strong> let’s create Microservices in the <strong>size and quantity that really fit our needs</strong>, no more, no less.</p><h3>Sources</h3><p>1997 — Steve Vinoski — <a href="https://www.google.nl/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwjNjcrh-a3SAhXMCywKHWFgAEwQFggdMAA&amp;url=http%3A%2F%2Fwww.cs.wustl.edu%2F~schmidt%2FPDF%2Fvinoski.pdf&amp;usg=AFQjCNGHgp0XUqdxj7IF5pKEGNr8KSop2A&amp;sig2=5dDa9AU9WVrxa7d64Ih88g&amp;bvm=bv.148073327,d.bGg">CORBA: Integrating Diverse Applications Within Distributed Heterogeneous Environments</a></p><p>2000 — Roy Fielding — <a href="https://www.google.nl/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=2&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwjxhZPc_K3SAhUFtRQKHSFJBUsQFggsMAE&amp;url=https%3A%2F%2Fwww.ics.uci.edu%2F~fielding%2Fpubs%2Fdissertation%2Ffielding_dissertation.pdf&amp;usg=AFQjCNEEwS3STct3jnKXToOQXO15Q4cY1g&amp;sig2=7B1V07RRh9X6vFTcFVRMVQ">Architectural Styles and the Design of Network-based Software Architectures</a></p><p>2004 — Microsoft — <a href="https://msdn.microsoft.com/en-us/library/ff647328.aspx">Message Bus</a></p><p>2004 — Microsoft — <a href="https://msdn.microsoft.com/en-us/library/aa480021.aspx">Understanding Service-Oriented Architecture</a></p><p>2011 — Chris Ostrowski — <a href="https://www.youtube.com/watch?v=0hyXOuvyq2Q">Understanding Oracle SOA — Part 1 — Architecture</a></p><p>2011 — Chris Ostrowski — <a href="https://www.youtube.com/watch?v=zNvyCUO0dyw">Understanding Oracle SOA — Part 2 — Technologies</a></p><p>2011 — Chris Ostrowski — <a href="https://www.youtube.com/watch?v=0J4iHaUOpzU">Understanding Oracle SOA — Part 3 — Development</a></p><p>2011 — Chris Ostrowski — <a href="https://www.youtube.com/watch?v=t9-kj1veqXk">Understanding Oracle SOA — Part 4 — Business Benefits</a></p><p>2012 — Prabhu — <a href="http://www.mudskipper-solutions.com/home/service-oriented-architecture-soa">Service Oriented Architecture — SOA</a></p><p>2014 — Martin Fowler — <a href="https://martinfowler.com/articles/microservices.html">Microservices</a></p><p>2014 — PWC — <a href="http://www.pwc.com/us/en/technology-forecast/2014/cloud-computing/features/microservices.html">Agile coding in enterprise IT: Code small and local</a></p><p>2015 — Udi Dahan — <a href="https://www.youtube.com/watch?v=CVc3d4hrH6Y">Messaging Architecture and Services Bus</a></p><p>2015 — Sam Newman — <a href="https://www.youtube.com/watch?v=PFQnNFe27kU">Principles Of Microservices</a></p><p>2016 — Kai Wähner — <a href="https://www.youtube.com/watch?v=fITFdDU5L9w">Microservices: Death of the Enterprise Service Bus?</a></p><p>2016 — Abraham Marín Pérez — <a href="https://www.infoq.com/news/2016/06/corba-removed-java-9">Java 9 Will Remove CORBA from Default Classpath</a></p><p>2016 — Oracle — <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/idl/corba.html">CORBA Technology and the Java™ Platform Standard Edition</a></p><p>2017 — Wikipedia — <a href="https://en.wikipedia.org/wiki/Distributed_object_communication">Distributed object communication</a></p><p>2017 — Wikipedia — <a href="https://en.wikipedia.org/wiki/Common_Object_Request_Broker_Architecture">Common Object Request Broker Architecture</a></p><p>2017 — Wikipedia — <a href="https://en.wikipedia.org/wiki/Enterprise_service_bus">Enterprise service bus</a></p><p>2017 — Wikipedia — <a href="https://en.wikipedia.org/wiki/Representational_state_transfer">Representational State Transfer</a></p><p>2017 — Wikipedia — <a href="https://en.wikipedia.org/wiki/SOAP">SOAP</a></p><p>2017 — Wikipedia — <a href="https://en.wikipedia.org/wiki/Service-oriented_architecture">Service-oriented architecture</a></p><p>2017 — Microsoft — <a href="https://msdn.microsoft.com/en-us/library/bb833022.aspx">Enterprise Architecture: SOA in the real world</a></p><p><em>Originally published at </em><a href="https://herbertograca.com/2017/11/09/service-oriented-architecture-soa/"><em>herbertograca.com</em></a><em> on November 9, 2017.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72fdb7dbebfb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-software-architecture-chronicles/service-oriented-architecture-soa-72fdb7dbebfb">Service Oriented Architecture (SOA)</a> was originally published in <a href="https://medium.com/the-software-architecture-chronicles">The Software Architecture Chronicles</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>