The Eye-Opening Value of Open Source on z/OS

At the recent GSUK conference, PopUp Mainframe presented a session on some of the most useful open-source tools we are using in our lab. Here’s a recap of the information we shared.

Our Journey With Open-Source Technology

Aiming to improve productivity, over the past year the team at PopUp Mainframe has embraced open-source tooling. Open source has been harnessed within the PopUp development team, helping us implement faster approaches, streamline our build process, and automate countless tasks. We have seen wide-ranging and sometimes surprising benefits of using open source with z/OS and now rely on open-source tools to develop, build, and test PopUp Mainframe product features. And, with customers requesting open-source options for their PopUps, we also share best practice to help them get the most from these resources.

In short, open source is important to us. Here, we outline the open-source tools we are leveraging, and the benefits.

ZTI (Z Terminal Interface)

ZTI is a Python application which gives you a full 3270 interface in your command prompt. It removes the need for a separate (commercial) TSO terminal emulator product, while giving you nearly the same green screen experience. ZTI is great for anyone without a TSO emulator, but it has a whole host of other capabilities too.

ZTI enables you to interact with z/OS through command line – running commands, connecting to the master console, and more. This is powerful because you can script these interactions and include them in applications or automation. We are using ZTI in our team to run z/OS commands and utilising it for master console management too.

CBT Tape

CBT Tape is a library of free software for the IBM mainframe MVS, OS/390, and z/OS environments. It has evolved over decades and continues to expand. Useful CBT Tape facilities we have recently leveraged include –

SHOWZOS – makes MVS control information displayable, such as OS info, software and storage config. This can be web-enabled too.
WHOSON – shows you who else is logged on to TSO at any time.
Xmit Manager – enables you to view the contents of xmit files on a laptop, without having to copy it onto a z/OS system first.

ZOPEN Community

The Zopen community offers a range of tools which have been ported onto Z to help make Unix System Services (USS) easier to use. The more like Linux USS becomes, the better, as it makes USS easier to use for mainframers, as well as making the mainframe world more familiar and “standardised” and therefore easier to develop and maintain by anyone. There are around 300 tools in the zopen community.

Ten of the zopen tools are officially supported by IBM and distributed free-of-charge with the IBM Open Enterprise Foundation kit. These are all available pre-installed on our product. A couple of examples –

Vim: While the Vi editor is available on USS, it’s not very user friendly, and now zopen gives us vim instead. The vim text editor reduces the learning curve for a non-mainframer to be able to make changes very simply.
Bash: The popular Bash shell gives much more functionality than the Bourne shell (the default on USS), such as auto-completion, which saves a lot of time and makes for a more rewarding coding experience.

While these may seem like small benefits, they do add up and can be the difference between a useable, productive interface, and a frustrating interface which continually slows you down.

Zowe

There are two main components of Zowe which we are actively using in our team: Zowe CLI and Zowe Explorer (a VSCode extension).

Zowe Command Line Interface (CLI)

Zowe CLI enables you to interact with the mainframe remotely and use common tools such as shell commands, bash scripts and build tools for mainframe development. It provides a core set of commands for working with datasets, USS, JES, as well as issuing TSO and console commands. It has a very small learning curve since it provides user-friendly help, giving you all options so you can get started without being an expert. By contrast, TSO commands are very brittle and not user friendly, particularly for someone who doesn’t have a mainframe background.

The Zowe CLI is very extensible too, and there are many plugins available which can be used easily by a non-mainframer, and new features are being released all the time.

Zowe Explorer

Zowe Explorer is an extension for VSCode which allows you to interact with your mainframe LPARs through VSCode. Once you are connected, you can view and interact with datasets, USS and jobs through your IDE. Zowe Explorer in VSCode allows you to ‘find’ objects simply, something which is not simple in ISPF. It also enables viewing of multiple z/OS environments at once, and the compare feature is very useful.

CI/CD Pipelines

We have built CI/CD pipelines to deliver mainframe changes, and we have seen the benefits of robust, reliable, and accelerated build, test, and deployment of code changes. One of our CI/CD pipelines looks like this –

The pipeline is orchestrated by GitHub Actions, and is triggered when code is checked in to GitHub
Build is performed using IBM DBB (which is not open source)
Unit testing is performed using open-source Galasa and COBOL Check
Artifactory is used to store the binaries
IBM Wazi Deploy (not open source) deploys the code to the target environment
For integration testing, we are using Galasa again, this time with Selenium, for end-to-end testing of the mainframe application.

Observability

We have built a mainframe observability solution using Prometheus and Grafana. This has saved us countless hours finding info and troubleshooting and is now a one-stop shop for the team to self-service information. Some dashboards we have built:

A near real-time list of which z/OS environments we have. With PopUps being so quick and easy to stand up, we create new environments and tear them down often. We now have ready access to the full list of PopUps and other VMs, and their status, including which version of z/OS is running, the status of z/OS, a view of some subsystems on each environment, backup details, and more.
License information is now available in one place with expiry dates and number of available licenses clearly visible and trustworthy.
A report to show the usage of DASD volumes, highlighting which ones have usage over a threshold. This helps us manage our lab more proactively instead of waiting for a problem to occur.

We continue to expand our portfolio of dashboards and it is paying dividends for us. The dashboards are also available for PopUp customers and gives them observability out-of-the-box.

Ansible

We also make extensive use of Ansible in our lab – and within our core product. We have built numerous playbooks to manage z/OS environments, including config, software installation, maintenance, and user management. For more details see here.

Where next?

Mainframe innovation is driven by growing demand and fresh ideas from both end-user organisations, vendors, and the wider community. The Open Mainframe Project and other open-source efforts are delivering valuable technologies, and we’re excited about making the most of these advancements and bringing them to our customers through the PopUp Mainframe product.

Automating the Heck Out of Every LPAR – with Ansible

In today’s enterprise IT landscape, automation and operational efficiency have become a fundamental strategic imperative. At the GSUK conference, the PopUp Mainframe session entitled “Automate the Heck Out of Every LPAR” showcased how automation, particularly using Ansible, is transforming mainframe operations.

Here’s a summary of our discussion.

Why Automation Matters

At the heart of the story, the strategic value of automation sits across three pillars: operational efficiency, customer value, and innovation. For PopUp Mainframe, automation aims to enable scalable, repeatable processes that improve both our internal lab management and customer-facing features and services.

Ansible as the Automation Engine

We presented Ansible as the cornerstone of our automation strategy. It is quickly emerging as a de-facto industry standard to support automating mundane tasks. Its simplicity, powerful z/OS modules, and alignment with industry standards make it an ideal choice for a variety of use-cases. Additionally, Ansible’s marketplace and ease of learning have helped the PopUp Mainframe team quickly understand, devise, build, and deploy our own automation solutions.

Overcoming Technical Challenges

Of course, implementing new technology is never a totally flawless exercise, and adopting Ansible in the PopUp lab wasn’t without its own hurdles. The team encountered issues with VM inconsistencies, z/OS versioning, and password management. These challenges were met with a robust architecture that included GitHub integration, secrets management via IBM Cloud Secrets Manager, and a centralized Ansible control node. Establishing a more robust approach to Ansible implementation needed some thought, but in the end gave us the solid foundation we needed.

Architecture in Action: PopUp Playbooks

The architecture spans both virtual and physical mainframes. Ansible Playbook source code is stored in GitHub, provides playbooks which can be executed remotely against any z/OS environment, and secured with secrets management. This setup ensures flexibility and security while maintaining control over deployments.

The PopUp Mainframe product includes a broad range of Ansible playbooks to help users and product administrators simply and accelerate routine tasks. Our playbook library covers the following areas, all intended to streamline and simplify the administration of the PopUp virtualized z/OS environment –

z/OS health checks
Configuration and user management
Maintenance routines
Software installation

These examples illustrate how automation can handle routine tasks reliably and consistently, and are just the start of our journey. We are constantly adding new playbooks to the product, which we will continue to share with our customers.

Automation Outcomes

The team’s reflections on their journey underscored significant improvements: increased standardisation, documentation of expert knowledge in code, and notable gains in operational speed. These outcomes not only streamlined routine procedures but also laid the groundwork for ongoing innovation and efficiency across the mainframe environment.

Improved standardization, codified knowledge, and speed improvements were all tangible outcomes of our automation journey and, with the resulting Playbooks now provided with the product, we’re delighted to provide the same benefits to our customers.

Tips for Implementation

Reflecting on our own journey towards automation using Ansible, for teams looking to adopt the technology, we would offer the following practical advice:

Leverage the Ansible community. There is a wide variety of support out there if you know where to look, including monthly Ansible community guild calls, an active Discord channel, blogs, LinkedIn groups, forums, and more.
Get team buy-in early. We ran hands-on playbook sessions where all team members were invited to login and run some playbooks. They quickly realised how easy it was!
Define standards (for example using Ansible lint) and write clear README files – this is critical to ensure the playbooks can be run without specific technical knowledge which is key to democratising access to z/OS.

Additionally, determining the right way to measure that progress helps build a tangible aspect to the journey that you can record, report and use to justify further steps.

Looking Ahead

Strategically, the path of continuous operational improvements is never complete. So, for our Ansible implementation, our future plans include:

Better tracking of playbook validation
Building additional custom playbooks for specific use cases
Making playbooks available to customers via AAP (Ansible Automation Platform)
Exploring event-driven automation

Here’s a previous article about the earlier steps in our Ansible journey. For more information on the PopUp Mainframe product, go here.

Look Back at Tech Field Day at SHARE, Aug 2024

Gary Thornhill, PopUp founder and CEO, had the privilege of presenting a fully featured PopUp demo to a group of mainframe and tech enthusiasts at Tech Field Day at SHARE Kansas City, 2024.

The delegates came from a wide range of backgrounds including appdev, mainframe education, security, DevOps, mainframe services, and Open Mainframe Project.

Watch the session here!

Gary jumped straight into the session with:

“What is PopUp? Well, we can popup a full mainframe platform in under 10 minutes, in the cloud or on prem.

We are proud that PopUp runs on IBM zD&T under the covers – tried and true industry-leading z/OS virtualisation software.”

Welcome to the Wonderful World of PopUp Mainframe!

PopUp was born out of a direct need to provide customers with mainframe environments so they can action the mainframe transformations they desire.

PopUp is pipeline-ready, and can be fully administered in a distributed way, using APIs to spawn, clone, and tear down a PopUp just like other virtual machines.

Using PopUp to deploy and test code quickly, test the latest security features, meet regulatory requirements, and test upgrades of Z can liberate your teams to achieve their goals more quickly than before.

Modern mainframes are incredibly efficient, cost effective, and provide better per-core performance than the alternatives.* Migrating workloads away from the mainframe requires careful consideration.

Using PopUp to augment your physical mainframe by enabling faster, cheaper testing on ephemeral environments is a fast track to modernising your mainframe processes and continuing to reap the benefits of the physical mainframe.

Gary discussed the key features of PopUp. Some of the call out points are:

We have developed a conveyor belt method to migrate data and config at speed from the physical mainframe to the PopUp, taking care of the SMS challenge.
PopUp provides system and helper utilities. For example, to help automate archiving, and CPU utilisation monitoring to assist in capacity management.
We have extended the Delphix Continuous Compliance engine to enable masking of ANY mainframe data source in a consistent, robust fashion across the entire organisational data estate, maintaining referential integrity across all systems, whether they be mainframe or distributed.

Using PopUp & Delphix for Data masking

PopUp Integrates fully with Delphix to utilise the industry-leading data masking capabilities against mainframe data.

Data can be masked on the physical mainframe, or on the PopUp, depending on your needs. Then we use Delphix Continuous Data self-service portal to provision compliant copies.

“Being able to rewind a full production-like mainframe environment at the touch of a button is absolutely massive for productivity and cutting time-to-market.

Marty McFly would be right at home with the 4D mainframe. Think of it as the ‘undo’ button for a mainframe.”

We have different flavours of PopUp. In this demo we show the PopUp & BMC, which comes with all the BMC DevOps tooling out of the box.

Getting Going with PopUp

We know implementing virtual mainframe environments might seem daunting, but don’t panic!

Our Delivery Manual is available to walk you through step-by-step how to get the most out of your PopUp.

We provide full product support with PopUp, and also manage anything to do with IBM zD&T.

Democratising Access to Mainframe

PopUp provides green screen for those who are comfortable with that, but it also enables users with modern IDEs and other tooling.

PopUp users can be productive quickly using their tools of choice, without any approval from the mainframe infrastructure teams or dependence on setup on the physical mainframe.

PopUp is a great way to train people new to the mainframe, as PopUp is a disposable, completely isolated environment with zero risk to the physical mainframe – perfect for training.

“With PopUp everyone is welcome and able to be productive with tools of their choice.”

Use cases on the PopUp are only limited to your imagination

Just some use cases for a dedicated mainframe environment:

Development
Use with a desktop IDE
All aspects of Testing including Regression testing against mainframe
Software evaluation
R&D
Proof of concept development
Offshore or 3rd party environment
Training environment
Use of a mainframe data “gold copy” on demand across the enterprise
Checkpoint, forward and rewind data

What we’ve all been waiting for… the demo!

Gary’s demo covered:

Spinning up a PopUp Mainframe in the cloud through the Azure MarketPlace. The PopUp can be ready to go in 6-8 minutes
Deploying and configuring an application using a GitHub Actions pipeline (covering tasks which would usually be done by a sysprog, automated testing of the UI through Selenium)
Masking the sensitive data through Delphix Continuous Compliance
Creating the baseline ‘Gold copy’ PopUp containing the masked data and full application
Enabling easy creation of on-demand virtualized PopUp instances with fast, efficient checkpointing
Provisioning compliant dev environment through the Delphix self-service portal
Developer commits code change into Git, pipeline is kicked off to generate, test and deploy the change onto a PopUp, then do UI testing
Failure of automated UI testing
Rewinding the entire PopUp environment back to the previous version (not just the data, the entire app) through the self-service Delphix portal
How a developer can understand what went wrong and fix it without any environment overheads

Some great questions from the audience…

Cameron asked if we need to install DB2. Answer: No, DB2, CICS, MQ, IMS and other apps come out of the box with PopUp and are available in 10 minutes.

Mark asked if there is an option for customers who can’t move unmasked data off the mainframe. Answer: Yes, masking can be done on the physical mainframe, and only the masked data ported onto the PopUp.

“Using PopUp allows organisations to easily start doing DevOps and automation on the mainframe in exactly the same way they do for their distributed applications today.”

Thanks to all the fabulous delegates who attended and joined in the conversation with energy and insight.

What a great experience! Thanks to the Tech Field Day team, in particular Stephen Foskett, for running such a great session.

Watch the session here!

Catch us at the upcoming GSE conference in November at Whittlebury Hall.

Platform Engineering: Building a Mainframe Internal Developer Platform (IDP)

“Platform engineering emerged in response to the increasing complexity of modern software architectures. Today, non-expert end users are often asked to operate an assembly of complicated arcane services,” says Paul Delory, VP Analyst at Gartner. “To help end users, and reduce friction for the valuable work they do, forward-thinking companies have begun to build operating platforms that sit between the end user and the backing services on which they rely.”

Source: gartner.com/en/articles/what-is-platform-engineering

The ultimate goal of platform engineering is to simplify and accelerate software delivery. By eliminating the burden of various tools, dependencies, and access restrictions, developers can zero in on what they excel at – building systems.

Across the industry, Platform Engineering is becoming as important as DevOps. In this blog, we delve into its transformative application within the mainframe space.

PopUp Mainframe & MooodyCow: Partnering for Faster, Cheaper, and Better z/OS Development

We are thrilled to announce that PopUp Mainframe are partnering with MooodyCow.

MooodyCow are a leading mainframe services provider. Their mainframe specialists cover all areas of mainframe, with a special focus on helping businesses modernise and optimise their mainframe systems.

Our combined vision at PopUp Mainframe and MooodyCow is to deliver cutting edge software and best practice know-how to help organisations achieve their ambitious mainframe optimisation goals.

To find out more, contact us here, or visit the MooodyCow site.

DevOps Excellence Awards 2024

We are very proud to announce that PopUp Mainframe has been awarded Highly Commended in the category ‘DevOps Tool/Product of the Year’ at the DevOps Excellence Awards 2024.

We are thrilled to be recognised alongside some well-established players in the DevOps space.

For many organisations, mainframe is often the last piece to achieve automation. Most businesses have established automation and DevOps ways of working for the distributed teams, but haven’t managed to apply these to the mainframe areas yet.

Adopting DevOps for the mainframe poses some unique challenges. These include a lack of dev and test environments in which to trial automation, new ways of working and new tools, as well as lack of innovation opportunity. Compounding this, there is a skills shortage in mainframe and sometimes resistance to change.

At PopUp, mainframe modernisation and mainframe DevOps is at the heart of everything we do.

Our mission is to empower mainframe businesses to deliver changes faster, cheaper and better with on-demand environments for dev and test. We see this as a necessary pre-requisite for organisations to move into this final ‘Automation & DevOps frontier’ (where ALL teams, including mainframe, can embrace automation and DevOps practices), and truly achieve their agility aspirations at all levels.

PopUp enables customers to take mainframe on their DevOps journey by creating fully functioning z/OS instances on-demand, on-prem or in the cloud, and controlling them through pipelines. This overcomes dev and test mainframe unavailability – the number one blocker to mainframe DevOps.

PopUp liberates mainframe teams, enabling automation, DevOps innovation and continuous improvement with no risk to the physical mainframe.

PopUp helps fast track DevOps initiatives in the following ways:

Aiding the implementation of automation and pipelines (e.g. with BMC tooling pre-installed on a PopUp)
Continuously improving ways of working and increasing collaboration
Helping to create real, lasting cultural change
Increasing productivity by reducing wait times
Modern tools boost the productivity of all team members
A catalyst for mainframe innovation and fast feedback
Ready-made cloud z/OS environments (e.g. Azure)
PopUp & Git integration is a cinch
Accelerating DevOps journeys with new talent

PopUp also provides additional features to boost your DevOps journey which are not available on the physical mainframe. For example, PopUp can be replicated to create multiple z/OS clones including applications and data, and mainframe instances can be deployed and torn down using APIs or pipelines. In conjunction with Delphix, mainframe instances can be bookmarked at any stage, then they can be rewound or fast forwarded to the snapshot using a self-service portal.

To learn more about how PopUp can help you achieve your DevOps goals, read the blog: Fast Track your DevOps Mainframe Initiatives.

Curious to know more about DevOps for the mainframe, how ephemeral mainframe instances can accelerate your projects, or automating your mainframe processes? Contact us.

Awards Cloud Culture Delphix DevOps Testing

PopUp Mainframe announces release of Warp Speed PopUp

Breaking news: PopUp Mainframe, the virtual, on-demand mainframe, today announced the general availability of their latest release: ‘Warp Speed PopUp’.

After much innovation, development and rigorous testing, we are proud to unveil the release of our new and improved ‘Warp Speed PopUp’ which has been massively optimised for fast installation and start up.

Our latest version of PopUp (version 2.4Ua) now boasts an installation and start up time of just 10 minutes. This represents an approximate 10-fold improvement on the previous version, which, whist no slouch, took between 45 and 90 minutes for installation and start up. Data compression has also been built-in at the storage level, resulting in a monumentally compact footprint for the Warp Speed PopUp which is now 5x smaller than any previous version of PopUp Mainframe.

The Warp Speed PopUp introduces innovations which enable businesses to further accelerate their mainframe modernisation projects. Lightning fast deployment of PopUp Mainframe instances improves the self-service user experience, greatly shortens the feedback loop and maximises environment availability for dev and test teams.

With BMC AMI DevX tools preinstalled and configured on the PopUp, and in conjunction with Delphix, users can achieve complete mainframe DevOps with the ability to snapshot, rewind and fast forward their mainframe to any point in time.

Get ready to revolutionize your mainframe software development and testing experience with Warp Speed PopUp – the future of on-demand mainframes is here!

“Being able to install and start up a PopUp in less than ten minutes is massive for customers. A team can go from having no extra z/OS capacity to have individual mainframe instances for each team member – in under 10 minutes! This is game-changing for mainframe dev and test teams.”

– Stuart Feasey, PopUp Presales Engineer

For full details on the latest PopUp version, see the PopUp documentation site.

About PopUp

PopUp Mainframe is a cutting-edge mainframe virtualisation product. PopUp provides IBM ZD&T software along with PopUp IP, which has been installed, configured and setup, and is ready to use out of the box. It enables customers to create mainframe environments on-demand and deploy them anywhere, including in the cloud.

Mainframe businesses often cite dev and test environment bottlenecks as the number one blocker to mainframe modernisation. PopUp directly addresses this, providing fully functioning mainframes via self-service, available on-demand. PopUp brings innovation to mainframe teams, providing features not available with a physical mainframe, for example the ability to take a full baseline of a mainframe environment, and replicate that using APIs. PopUp, in combination with Delphix, makes it possible to have a 4D mainframe: users can take a snapshot of a mainframe environment, then rewind or fast forward to a point in time. For an instant DevOps mainframe, BMC AMI DevX tools come shipped with a PopUp – preinstalled and configured, ready to go.

With PopUp, mainframe businesses attract a new generation of mainframe engineers who can use cutting-edge mainframe tooling with PopUp, for example desktop IDEs. PopUp simultaneously appeals to traditional mainframe engineers who prefer green screen and now have the autonomy to spin up their own mainframe environments on-demand. PopUp runs ANY mainframe subsystem or bespoke application.

Old School, New School … Mainframe is for Everyone!

At PopUp, we believe mainframe can and should be for everyone.

Let’s start this post with a definition.

‘Old School’ Mainframe Engineers: People who may have been working on the mainframe for many years who are comfortable with green screen and traditional ways of working.

‘New School’ Mainframe Engineers: The next generation of people who want to utilise modern tooling and work in collaborative agile ways.

Of course the situation is not black and white, and there are many mainframe engineers who fall somewhere between these two groups. But these phrases will be helpful for discussion.

The ‘New School’ perspective on the mainframe

For some ‘new schoolers’, mainframe is seen as an old, outdated technology which is not required in the modern world. They may also think that a career in mainframe is not a wise choice due to the perception of mainframe usage reducing, therefore providing less opportunity to grow and develop new skills. However this is far from the truth of the situation.

The results from the recent 2023 BMC Mainframe Survey Report tell us that year on year, more and more people working in mainframe, and particularly at the executive level, believe in the mainframe’s ability to grow and attract new workloads. This perception is growing, and reflects a steady rise in data volumes and workloads on the mainframe in recent years.

Mind the Skills Gap

Unfortunately, we are in a situation where many (‘old school’) mainframe developers are retiring from the workforce leaving behind a workforce skills gap, and in many cases, extensive and difficult-to-support code libraries.

To address this skills shortage, companies need to attract the next generation of mainframers (‘new schoolers’) to their teams. To be attractive, organisations must provide modern mainframe development tools and embrace modern ways of working (which are usually appealing to ‘new schoolers’). Although many people love trusty ISPF, it’s just not sexy!

‘New schoolers’ need to see that mainframe is a dynamic, relevant platform which is here to stay for the long term, with a wide enough scope to sustain a long career.

For new hires to be effective, ‘old schoolers’ and ‘new schoolers’ need to work side by side collaboratively and productively on the same projects. Due to the different working styles and expectations, some organisations are struggling to get these groups of people to work effectively together.

It doesn’t matter if you’re ‘old school’, ‘new school’ or somewhere in between, PopUp can help!

PopUp Mainframe is a z/OS virtual instance running real z/OS code for Dev and Test, identical to a physical mainframe. PopUp comes packaged as a fully featured, optimized virtual machine image. It runs on any enterprise x86 hardware you choose – on prem or in the cloud.

PopUp helps attract ‘new schoolers’ to your organisation, while supporting ‘new schoolers’ and ‘old schoolers’ to work together and improving team productivity. It does this in many ways:

With PopUp, ‘new schoolers’ can easily use automated pipelines and APIs. PopUp also makes it easy for teams to adopt modern ways of working introduce automation pipelines and start using git. This is very attractive for ‘new schoolers’ and may aid recruitment.

‘New schoolers’ will be at home using a PopUp, as PopUp ships with modern tools already installed set up and ready to go (meaning nothing needs to be installed on the physical mainframe). ‘New schoolers’ can use IDEs which provide remote debug and editing, code assist and more.

Using PopUp, ‘old schoolers’ are free to use green screen and traditional mainframe tools, which look exactly the same on a PopUp as they do on a mainframe (since ZD&T is a hardware emulator and emulates the Z hardware).

PopUp levels the playing field for ‘old schoolers’ and ‘new schoolers’ by offering a range of tools with which to perform development activities and enabling each team member to choose which tools they are most comfortable with (while working side-by-side on the same projects).

This strategy of providing a range of tools to team members and letting them choose which ones they are most comfortable with also brings other benefits. It supports ‘old schoolers’ to adopt new ways of working and new tools at their own pace. It is likely to improve job satisfaction as team members have some autonomy about their tool usage.

Organisations can recruit new team members who prefer ‘old school’ ways of working, ‘new school’, and anything in between. New hires will be quickly productive either way.

Together we all can (let’s be confident!) … will… change the perception of mainframe as a legacy technology.

An Example

One of our customers, a pensions company, supported their team members (which consisted of ‘old schoolers’, ‘new schoolers’ and some people in between) to transition from green screen to modern mainframe tools in a phased approach. They did this in small groups, by giving each person a PopUp and supporting them to use modern mainframe tools when each person was ready – at their own pace and comfort level. This approach was driven by empathy and resulted in a very successful project and productive team.

As this customer said,

“From going from a green screen, to having an IDE…that’s just a complete game changer.”

If you are curious to know more about how PopUp can support your mainframe modernisation goals, contact us today.

Culture DevOps

Masking mainframe ‘legacy’ data sources – Tricky or trivial?

Data protection and compliance is such an important topic, and there’s no exception on mainframe. But it can be complex to implement for mainframe, particularly in relation to what we sometimes term as “legacy” mainframe data sources. This blog post discusses why it is difficult, and how data masking on legacy mainframe data sources can be done.

In “modern” mainframe databases such as DB2, metadata is readily available to describe each and every field within a table. This data is typically held within a set of system tables that can easily be referenced by applications. In the case of DB2, this information is collectively known as the DB2 Catalog.

Jump back 40 years (yes, frightening isn’t it!) or more, and the situation is not so straightforward.

Legacy data structures required ‘Data dictionaries’

Users of what many of us consider to be primitive data structures, such as fixed and variable length flat files, VSAM files and early databases such as the hierarchical IMS and Networked IDMS, did not have such an easy life. Data dictionaries were developed to keep track of data usage within such table structures. But there was still a dependence on external language dependant data structures such as COBOL COPYBOOKS and PL/I INCLUDE members, together with the necessity for often manual scanning of the (tens of thousands of) application programs that might access these data sources, to get a full understanding of exactly which record contained which data and at which point in time.

Despite some organisations having whole “Data dictionary” teams of former developers and designers assigned to keep these devices up-to date, often such activity fell outside of the organisations change management process, and led to issues down the line.

Modern data privacy requirements are myriad and strict

With the introduction of compliance initiatives such as GDPR and CCPA, the processing of data sources within organisations become a major focus. In production, access to the data is restricted to only those who need it. But in test environments, the story is more complex. Production data (‘real’ data which covers valid scenarios) is required in test environments to enable high quality testing, but the data must remain secured to ensure data compliance.

Data obfuscation, or masking to those of us who cannot pronounce it (you know who you are), needs to be applied to ALL data sources that hold personally identifiable information (PII). In the mainframe teams, this covers DB2, IMS, IDMS, VSAM, flat files and even MQ messages, etc.

Modern day metadata needs to be acquired, unravelled, documented and verified so that masking can be performed adequately to protect the identity of every individual. Notably, masking also has to be performed in a simple and consistent way across an entire organisation’s estate – including mainframe and distributed platforms.

How to mask legacy data structures in a simple and consistent way?

Delphix Continuous Compliance Engine provides excellent data compliance capabilities for most data types. In conjunction with Delphix, PopUp’s z/OS Masking Plugin for Delphix, provides a simple solution for masking legacy mainframe data sources consistently.

With the PopUp masking plugin, VSAM files, IMS databases and data from any mainframe database (that can be unloaded to/reloaded from flat files) can now be processed for masking. COBOL copybooks and PL/I Includes can be processed effortlessly, allowing the Delphix Profiler to work its magic and identify and allocate algorithms based on the data formats.

Data residing in these ‘legacy’ mainframe data sources can be processed by Delphix Continuous Compliance (via the PopUp masking plugin). This means that data from sources across an organisation can be represented and treated (masked) in the same way, reducing the need for specific mainframe expertise.

For more information on the z/OS Masking Plugin for Delphix, watch our video.

If you have data compliance requirements for legacy mainframe data sources, we can help! Get in touch to hear more.

Data Masking Delphix

Fast-track your mainframe DevOps initiatives

DevOps on the mainframe is particularly hard

In IT, mainframe is often the last piece to achieve automation. Many organisations have established automation and DevOps ways of working for the distributed teams, but haven’t managed to apply this to the mainframe areas yet.

As we see it, the main challenges in adopting DevOps for mainframe are:

Lack of dev and test environments in which to trial automation, new ways of working and new tools.
Culture, mindset, and ways of working changes (and ensuring those are sustained in the long term).
Inability to innovate due to locked down systems and timeboxed access to the physical mainframe.
Governance processes which are entrenched and inflexible.
Mainframe skills shortage and difficulty attracting new joiners.
Create a right-sized virtual z/OS instance with the right data and applications to suit the use case (not just a full copy of the LPAR).

We developed PopUp to help solve some of these pressing business problems.

At PopUp, our mission is to empower mainframe businesses to deliver changes faster, cheaper and better with on-demand environments for dev and test. We see this as a necessary pre-requisite for organisations to move into this final ‘DevOps frontier’ (where ALL teams, including mainframe, can embrace DevOps). This is necessary for businesses to truly achieve their agility aspirations at all levels of the business.

Find out more information about PopUp here.

How does PopUp fast track mainframe DevOps initiatives?

PopUp enables an organisation’s mainframe DevOps initiatives to take off, in a number of ways. PopUp enables companies to eat the elephant of mainframe DevOps by starting small and making progress quickly.

A. Pipelines & Automation

Automation and pipelines are a must for any DevOps journey. PopUp aids the implementation of automation and pipelines by:

Enabling users to spin up and manage fully functioning z/OS instances through a pipeline via RESTful APIs.
Making it easy to align mainframe pipelines with pipelines for distributed systems.
Providing mainframe teams with automation tools pre-installed and pre-configured on the PopUp.
Using the built-in test automation tools, developers can write unit tests and run them through automation in a PopUp.
Providing test automation tools which can be used to build a full regression test pack.
Making it easy to include code validation in pipelines.
Integrating out of the box with Git.

B. Increase Productivity by Giving Developers Mainframe On-Demand

The main bottleneck in mainframe delivery is lack of mainframe environments. This causes a huge knock-on effect when teams have limited slots on the physical mainframe, and have to schedule those far in advance. PopUp delivers z/OS environment capacity to teams at the speed its needed. This newfound availability of mainframe instances in turn enables fundamental changes to the teams’ ways of working.

Team members no longer need to wait days or weeks for their slot on the mainframe, instead they can do their work as needed. Once the biggest blocker is removed, wait times and idle time are reduced through every stage in the delivery process. This has a massive impact on productivity, as well as team satisfaction.

C. Modern tools boost the productivity of all team members

There are more and more tools available which allow developers to interact with mainframe systems without ever logging into a mainframe green screen. Modern mainframe tools provide a host of benefits including intuitive, easy-to-use interfaces, automatic code completion and syntax checks, debug tools, automation, and instant feedback.

However, it’s not always easy to adopt modern tools on the physical mainframe. On PopUp you are free to install or use any modern mainframe tools you wish and experiment in an isolated, completely risk-free environment. Mainframe teams can then realise the benefits of modern mainframe tools with no risk to the physical mainframe. Modern tools not only improve productivity of new joiners (who can be very productive using intuitive tooling even without prior mainframe experience), but vastly increase the productivity of existing team members too, who experience the same benefits.

D. DevOps with Compliant Data

One challenging element of ephemeral environments is getting the right data. Using PopUp in combination with Delphix, production data can be masked and moved onto the PopUp. This gives developers and testers the ability to test against fully compliant production-like data. The benefit of using prod-like data for testing is hard to overstate – it facilitates better test scenario creation and increased test coverage.

Another powerful feature of Delphix & PopUp is creating a Gold Copy image which includes masked data. This Gold Copy can be cloned as needed to provide a full z/OS instance with compliant data, ready to go. The test data can also be refreshed easily, ensuring that dev teams always have recent, prod-like, compliant data.

E. PopUp is a catalyst for mainframe innovation and fast feedback!

PopUp brings increased opportunity for innovation to mainframe teams since they can work and experiment in 100% safe, disposable environments. In a risk-free mainframe sandbox, you can be confident in experimenting with new capabilities which cannot be risked on the physical mainframe. This gives the opportunity for fast feedback on new tools and capabilities without needing to install anything on the physical mainframe.

Using Delphix alongside PopUp allows users to provision copies/clones of a PopUp, snapshot the instance during testing, then just rewind / forward-wind to any of the snapshots as needed. There is no need to worry about breaking the PopUp. This gives app teams tremendous flexibility and independence to develop, test and deploy changes faster.

F. Ready-made cloud z/OS environments

PopUp is available to buy directly through Azure MarketPlace and can be up and running in less than an hour. This provides near-instant z/OS compute capability in the cloud to service DevOps initiatives.

PopUp is also compatible with other cloud environments.

G. Accelerate your DevOps journey with new talent

Skills shortage is a big challenge in the mainframe space and can derail DevOps initiatives. Mainframe businesses need to attract a new generation of mainframe engineers, who usually want to use cutting-edge mainframe tooling, like desktop IDEs and automation tools.

Modern mainframe tools come out of the box with PopUp. So PopUp simultaneously appeals to traditional mainframe engineers who may prefer green screen (and enjoy the ability to spin up their own mainframe environments on-demand), as well as new mainframe engineers who prefer modern tools. Using PopUp, you can hire people with no green screen experience and be confident that they will still be able to integrate with and develop mainframe systems.

H. PopUp – Git integration is a cinch

Most DevOps initiatives rely heavily on modern version control tools. Git is regarded by many as the best, easiest to use, and the most effective one available and it’s everywhere in the world of distributed applications – but not yet for mainframe. PopUp enables Git integration with mainframe source out of the box, giving yet another boost on your DevOps journey.

How does PopUp help to create real, lasting DevOps cultural change?

This is often one of the trickiest parts of a DevOps transformation journey. It can be particularly difficult in mainframe teams which may have been operating in similar ways for decades and have no appetite to change.

It’s easier to make incremental cultural changes than forcing a completely new way of working in one go. PopUp provides the vehicle for this as users can adopt PopUp at their own pace alongside their existing ways of working on the physical mainframe.

PopUp empowers users as they benefit from the self-service capability, and that results in appreciation and uptake from the mainframe teams.

PopUp gives extra features to boost you on your DevOps journey

One amazingly powerful thing about PopUp is that it provides capabilities which physical mainframes cannot. This opens the door for deeper innovation and new working practices:

Replicate PopUps to create multiple z/OS clones including applications and data
Deploy and tear down mainframe instances using APIs or pipelines
Baseline and version PopUp instances
Take snapshots of PopUp throughout projects using Delphix
Return to a baseline (rewind), and fast-forward to snapshots
Switch a PopUp off when it is no longer needed. When it is brought up again it is exactly as it was – there are no ongoing maintenance costs as you have for a physical mainframe.

Curious to know more about mainframe DevOps? Contact us to discuss.

Cloud Culture Data Masking Delphix DevOps

Solution

Products

Resources

About