As I’m saying often, scripts are the level zero of the software development and integration. By scripts, I’m reffering to these sequences of command lines ran by the Unix shell. And when I say that they represent the level zero of the software, I don’t mean to deny their utility to automate simple processes like copy a set of files, zip/unzip archives, deploy/undeploy stauff, etc. What I mean is that, while they might be practical ways to quickly perform repetitive actions, like checking-out dozens of projects from an SCM (Source Code Manager) repository, their utilisation for important things, like infrastructure management, should really be forbidden.

I’m currently working in a DevOps team of a big french gouvernment administration, where the infrastructure is quite impressive, consisting in more than 600 platforms, each one having several dozens of WebLogic domains.  And as surprizing as it might seem, all the infra-structure services consisting in installing binaries, configuring domains, application servers, databases, clusters, web servers, deploying applications, etc., is done in shell scripts having thousands of lines. These scripts look like a massive spaghetti pile, having often several thousands of lines, without any structure or clear defined interface and where evrything is done via sed and awk. And of course, to cap it all, nobody in the team has the mastership of what exactly happens in those scripts.

Different consultants, including myself, have tried to sensibilize the management to the huge technological debt they will have probably to pay for a very longtime, thanks to the choices they made and, while they become aware of this situation, they still don’t seem to be at 100% convinced. They don’t understand what’s the point as far as everything seems to almost work.  Different approaches are in-progress trying to introduce different open-source DevOps virtualisation and install engines solutions, like Vagrant, Ansible, etc. But the effort is far from being trivial as this requires to dig in thousands of lines of spagetti like scripts, to extract functionality and to think how it could be re-designed and re-implemented with tools like ansible or equivalent.

However, as far as I’m concerned, I’m still not confortable with this approach as replacing a scripting language, like the Unix Shell (Bourne Shell, C Shell, K Shell, BASH or whatever) by an automation engine, like ansible, doesn’t fondamentally change anything. While a more modern solution then using poorly designed and untested in-house scripts, while supported and professionally documented, an automation engine like ansible is not yet the tool to be used in order to implement SOA infrastructure services. As highlited by the SOA advocates, the infrastructure services are an important part of the enterprise services set and as such, they have to satisfy the following requirements :

• Modularity and granularity. In an SOA based infrastructure, services are modular and self-contained. They may be composed from other modular services, and can be mixed and matched as needed to create new composite services. Granularity is a critical quality for services—the more coarse-grained a service is, the richer or larger the function offered by the service. Coarse-grained services provide a greater level of functionality within a single service operation. This helps to reduce complexity and network overhead by reducing the steps necessary to fulfill a given infrastructure activity. Often this is accomplished by composing smaller tasks into a single coarse-grained operation. Fine-grained service operations provide the exchange of small amounts of information to complete a specific discrete task. These requirements of modularity and granularity cannot be achieved in scripting languages as they are too primitive and too closed to operating system and their commands and utilities to provide so high level concepts.
• Encapsulation. Services exhibit a strict separation of the service interface (what a service does) from the service implementation (how it is done). Encapsulation hides the service’s internal implementation details and data structures from the published interface operations and semantic model. Encapsulation is impossible to be achieved in scripting languages because they don’t have the notion of interface and because implementation is all what the scripts are about. Consequently, scripts don’t expose any semantic element.
• Loose coupling. Coupling describes the number of dependencies between a service consumer and provider. Loosely coupled services have few, well-known and managed dependencies. Tightly coupled services have many known and, more importantly, unknown dependencies. The degree of coupling directly affects the flexibility and extensibility of a system. From this point of view, it’s obvious that scripts can only be tightly coupled and, hence, unflexible and unextendible.
• Isolation of responsibilities. Services are responsible for discrete tasks or the management of specific resources. A key characteristic of service design is the isolation of responsibility for specific functions or information into a single service. This provides one (and only one) place for each function to be performed, providing consistency and reducing redundancy. But when you look at how scripts are written, even when they are written by experienced people, having a real and solid culture and expertize in the services world, you notice how far you are from notions like isolation of responsibilities.
• Autonomy. Autonomy is the characteristic that allows services to be deployed, modified, and maintained independently from each other and the solutions that use them. An autonomous service’s life cycle is independent of other services. As opposed to this concept, a script is something which heavily depends on the context it was written for and which doesn’t provide any designed to change feature or ability.
• Reuse. Together, modularity, encapsulation, loose coupling, isolation of responsibilities, and autonomy enable services to be combined into multiple processes or accessed by multiple service consumers from multiple locations and in multiple contexts. In other words, services are shared and reused as building blocks in the construction of processes or composite services. This is absolutelly not the case of the scripting technology which doesn’t provide any of these advantages.
• Dynamic discovery and binding. Services can be discovered at design time through the use of a design-time service repositories. Additionally, service consumers can be dynamically bound to providers during run time. In this scenario, the consumer asks the registry for a specific service and is routed and bound dynamically to the appropriate service provider. The dynamic binding of a service consumer to the service provider  enhances loose coupling and enables additional capabilities such as mediation. Once again, using scripting languages, we are very far from this scenario.
• Stateless. Service operations are stateless. This means that they neither remember the last thing they were asked to do nor care they what the next is. Services are not dependent on the context or state of other services — only on their functionality. Stateless services provide better flexibility, scalability, and reliability. Again, when you look at scripts where nothing is stateless, which depend on lots of things, including but not limited to the current folder in the file-systems, it becomes clear that they aren’t share the same play-ground.
• Self-describing. The service contract provides a complete description of the service interface, its operations, the input and output parameters, and schema. The contract may also contain pre and postconditions and constraints about the operations. In the case of scripts, there is no any contract, any interface or any schema. The well behaved scripts use to provide a –help switch explaining the required parametters and their meaning, but this is their maximum. They are not only not auto-describing but even not discribing at all.
• Composable. Services can be composed from other services and, in turn, can be combined with other services to compose new services or processes. Whoever ever tried to compose scripts or to chain them knows how difficult this exercice is. Besides sourcing scripts defining environment variables definitions in the context of other scripts, any other attempt of superior composition or aggregation is quite impossible, given their un-flexibility, their statefullness, their un-modularity and their un-autonomy.
• Governed by policy. Relationships between service consumers and providers (and between services and service domains) are governed by policies and service level agreements (SLAs). Policies describe how different consumers are allowed to interact with the service. In other words, what they are allowed to do. Scripts are at ages from such a processing model hence they are not mature enough to be used in SLA based environments and those having to guarantee a given quality of service (QOS).
• Independent of location, language, and protocol. Services are designed to be location-transparent and protocol/platform-independent. In other words, they are accessible to any authorized user, on any platform, from any location (within reason). As opposed to them, scripts require the user to know by hard the 120 characters long path on the file systems they are stored, together with SSH connection credentials and other horrors that make them simply unsable.

My conclusion is that, unless you need to automate simple processes consisting in running a sequence of OS commands, or even more complex processes like saving and restoring data bases or file-systems, in your daily infrastructure tasks you need to define discrete functionality units and to make them available through a contract, while making sure they are modular and in the right granularity, loose coupled, autonom, reusable, stateless, self-describing, etc. Meaning that you need to forget scripting languages. Yes, this is true even for scripting languages like Python, Jython, etc. And what other technology is the most appropriated for that ? Well, I know this is hard to be accepted by system administrators, infrastructure architects, DBAs and other production and operations engineers but, as unbeleivable as it might be, the technology you’re looking for is Java.