At Nearform we've had the privilege of working on a variety of projects, ranging from building apps from the ground up to critical interventions in ongoing developments. This extensive experience has given us deep insights into performance optimisation. We frequently collaborate with clients who demand exceptional scalability and performance, and our proven strategies have consistently delivered outstanding results.

Lately, many developers have made Next.js their web development framework of choice. This is due to its ability to enhance SEO and improve search engine rankings, driving more traffic and engagement to their website.

The rise in popularity of Next.js can be attributed in part to its robust performance capabilities. With built-in server-side rendering (SSR) and static site generation (SSG), Next.js empowers developers to create blazing-fast web applications without compromising on functionality or user experience.

However, even the most powerful tools require fine-tuning to unleash their full potential. This guide will cover general optimisation techniques, with a specific focus on Next.js, to help you achieve superior performance and better rankings.

As we dive into the details of optimising Next.js website performance, it's essential to recognise that speed is not just a technical concern — it's a fundamental aspect of user satisfaction and engagement. Users expect websites to load quickly and seamlessly, regardless of the underlying technology stack, these are some of the things to be aware of:

• 47% of users expect a page to load in under 2 seconds

• 100ms in latency costs 1% in sales

• Load times over 1s for a mobile site increase visitor bounce rate by 90%

While our focus is on Next.js, it's worth noting that many of the performance optimisation techniques we'll discuss can be applied across various technology stacks — irrespective of the framework, the principles of performance optimisation remain largely consistent.

Drawing from our past experiences, we've developed a checklist that can serve as a go-to guide for website optimisation. This checklist encapsulates some of the essential steps every developer should follow to ensure their website performs at its best. While we include general optimisation techniques, we also provide specific insights for Next.js.

Here are some quick tips from our checklist to get you started, and remember, our team at Nearform is always ready to provide expert guidance tailored to your specific needs.

Before delving into the intricacies of performance optimisation, it's essential to establish a robust monitoring system tailored to your application. Effective monitoring not only allows you to track the impact of optimisations over time but also facilitates the identification of potential bottlenecks, empowering you to make informed decisions for further performance enhancements.

For real-time monitoring and visualisation, leveraging tools like Prometheus and Grafana is paramount. Prometheus excels in collecting metrics from both your application and infrastructure, while Grafana offers comprehensive visualisation and analysis capabilities. Together, they enable you to scrutinise crucial performance indicators and help with identifying performance bottlenecks and resource constraints, paving the way for targeted optimisations.

In addition to real-time monitoring, investing in performance monitoring tools like NewRelic or Datadog provides invaluable insights into your application's performance at a deeper level. By analysing this data, you can fine-tune your application's performance, ensuring seamless user experiences across various scenarios.

Caching is one of the most effective ways to improve the performance of your web application. When implementing caching, it's essential to identify what resources and data should be cached, how long should those be cached for and when the cache should be refreshed.

When dealing with large volumes of data, slow performance can be a common issue. Even fetching cached data can unexpectedly take a significant amount of time. Our experience reveals the efficacy of compressing data (utilising formats like gzip or brotli) before caching, as it can notably enhance load times. It's worth noting that certain caching solutions offer this compression option, presenting a potential game-changer. Keeping a vigilant eye on such features could yield significant performance improvements.

Connection reuse is a crucial strategy for optimising the performance of web applications. By reusing existing connections, instead of establishing new ones for each request, you can reduce latency, and improve throughput and the overall speed of your application. This is particularly important for applications that make numerous HTTP requests or database queries.

Reusing connections eliminates the need for repeated Transmission Control Protocol (TCP) handshakes, reducing the time it takes to establish a connection and start data transfer while also reducing the computational and memory overhead associated with establishing and tearing down connections, leading to more efficient resource utilisation.

Ensure that your web server is configured to support Keep-Alive connections. If your application makes frequent database queries, consider using connection pooling to reuse database connections. When making HTTP requests from your Next.js application, opt for HTTP clients that support Keep-Alive. For instance, Node.js's HTTP and HTTPS modules inherently support Keep-Alive, while some third-party libraries, such as axios, may require explicit configuration for this feature. Set an appropriate timeout value to strike a balance between reusing connections efficiently and avoiding the risks of keeping idle connections open for too long.

Content Delivery Networks (CDNs) play a vital role in improving the performance and reliability of web applications by distributing content closer to users.

Select a CDN provider that suits your needs. Popular options include Cloudflare, AWS CloudFront, Akamai and Fastly. Each provider offers different features and pricing models, so choose one that aligns with your performance and budget requirements. Enable compression on your CDN to further reduce the size of transmitted data. Most CDNs support Gzip and Brotli compression. Use HTTP headers to control how and when your assets are cached.

The __NEXT_DATA__ object is a global variable injected by Next.js into every page on the client side to help with faster rendering and it can grow in size extremely fast (read more).

Often times in cases when Next.js apps have performance issues, __NEXT_DATA__ is a good place to focus for performance improvements. While the __NEXT_DATA__ object in Next.js is a powerful tool, misusing its capabilities can lead to serious performance penalties. The key thing here is to only keep crucial and essential data about the page in this object.

Try to review the data and remove any piece of data that is not needed for the initial rendering. We’ve seen huge improvements just by double-checking for duplicate data or removing certain properties that weren’t needed initially (and you definitely have some of those).

Lighthouse, is an open-source automated tool for measuring website performance and quality. It often flags images as significant performance issues, especially on mobile. Common problems include improperly sized and unoptimised images. To address these issues, the Next.js next/image component is highly effective:

• Automatic optimisation: Images are optimised based on device characteristics, reducing bandwidth and improving Cumulative Layout Shift (CLS) metrics.

• Lazy loading: Images load as they enter the viewport, enhancing initial page load times.

• Modern formats: Supports WebP and AVIF formats, which offer better compression and quality.

• Responsive images: Automatically selects appropriate sizes for various screen sizes.

• Placeholders: Provides visual feedback during loading, improving perceived performance.

Maintaining an up-to-date Next.js application is crucial for ensuring optimal performance. Regular updates bring numerous benefits, including performance enhancements, security patches, and new features.

Each Next.js release often includes optimisations that enhance performance. These updates can reduce build times, improve server-side rendering speed, and enhance client-side performance.

Eliminating unused JavaScript (JS) and CSS is a key strategy for optimising the performance of your Next.js application. Removing unused code reduces the overall size of your application, leading to quicker load times and a more responsive user experience.

Metrics such as First Contentful Paint (FCP), Time to Interactive (TTI), and Largest Contentful Paint (LCP) improve when there is less code to parse, compile, and execute. Smaller files are more efficient to cache, making repeated visits faster and reducing server load.

What can you do? Analyse your bundle and get rid of unnecessary code; make sure the project is set up to use tree shaking, dynamic imports can help in certain situations.

Every project is unique and fine-tuning an application’s performance will be unique to that application. The above list serves as a guide to follow to shave off some big wins when it comes to performance tuning. The most important thing is to have a strategy and improve the performance piece by piece.

Microsoft’s Semantic Kernel is a software development kit (SDK) that integrates Large Language Models (LLMs) with programming languages like C#, Python, and Java. The SDK does this by allowing us to define programming functions as plugins that can be easily chained together in a few lines of code.

Another major feature of Semantic Kernel is its ability to automatically orchestrate (or coordinate) plugins using AI. Simply provide the Semantic Kernel with an LLM and the plugins you need, then create a planner. When you ask the planner to achieve a specific goal, the Semantic Kernel will execute the plan for you.

An AI agent is a software program designed to perform tasks autonomously to achieve predetermined goals. It interacts with its environment (such as code functions, APIs, and data sources) and independently determines the best tools and actions to use for each task. AI agents can adapt to changes in their environment, learn from experiences, and improve their performance over time.

In summary, the user provides the goal which the AI should achieve and the AI will determine and execute the best approach to achieving that result.

We utilised Semantic Kernel to create an AI agent. The aim of the agent is to retrieve financial data, calculate the drawdown of the data, and finally plot both the data and the drawdown.

The above diagram shows how the agent communicates with the required APIs and functions to perform the desired tasks. A step-by-step explanation is available below:

1. The user asks a question via the terminal: User: give me details on Apple over the last 6 months

2. The agent will process the query to retrieve the required fields from the user’s query:

a. ticker_name: AAPL

b. period: 6mo

3. The agent will call a function to retrieve the historical data from the Yahoo finance API. This data includes a variety of information on the stock over the period provided. The values we are interested in are: date and close

a. date: the day the stock values were at the level indicated

b. close: the value of the stock at the time of closing of the financial markets on that day

4. The agent will call a function to calculate the drawdown

a. The drawdown is a financial measure that looks at the price of a stock at a certain point in time, gets the previous max value the stock held, and subtracts the max from the current value. The drawdown is a negative score as the current value is never greater than the cumulative max value up to the date we are looking at. The closer the drawdown is to zero, the closer the stock is to its maximum value.

5. The agent will then generate dynamic Python code to plot the data in a valid format using pandas and matplotlib.

6. Finally, the agent will refactor the code to attach the data calculated earlier on to the code just generated, then execute it

At the time of writing, Semantic Kernel allows four types of agents, called planners.

Sequential planner takes a goal (prompt) and runs through the plugins provided to the agent at creation time, in the order requested by the goal. The loop will continue until the goal is complete. This is the agent we used to accomplish our goal.

The process our planner follows is shown below:

An agent that will run through the user request and process it step by step. It differs from the sequential planner by only running a specific agent against a part of the query. This allows the agent to break down the query into individual parts and run those against the plugin.

An example is if we asked “What would be the temperature in New York if we added 10 degrees to the current temperature?” the stepwise planner will be able to split the weather API call and the math problem from the query and run them separately, giving answers based on them.

The main advantage of the handlebar planner over the stepwise planner is the usage of the handlebar templating language to generate the plan. This is due to the majority of LLMs already supporting the handlebar interface when dealing with prompts, which helps provide improve the accuracy of the response (in this case the agent plan). In opposition to the stepwise planner that uses XML when generating the plan. Using this language we can also provide loops and conditions that otherwise would only be available in coding languages.

Another advantage of this planner is the fact that the return value of the serialised function planner.getPlan is a readable text file that can be stored and reloaded at a later date. However, this planner is not available in the Python version of the kernel, as of the time of writing.

Semantic Kernel offers us the ability to create our agent with custom rules. This involves more complexity but also gives more freedom to the developer to create the agent in the way they intend. It is particularly useful when you want human intervention during the agent planning stages, for example, to provide new variables or access custom data sources.

At the time of writing, the team behind Semantic Kernel intends to deprecate the Handlebar and Stepwise planners, in favour of this approach.

The below code demonstrates how we created our agent using the Sequential planner:

1. Create the kernel Instance: This is the starting point to map our planner to the plugins and LLM. We can generate more than one planner with the same kernel, allowing for parallel agents if needed.

2. Configure AI service used by the kernel: This allows the tool to interpret user input into the right parameters and ensure the right tool is called at the right time to accomplish the goal.

3. Adding our custom plugins to the kernel: This enables non-standard LLM calls to be usable by our agent

4. Define the details to be passed to the planner: These include standard LLM settings (temperature, token limit, …) and function handling instructions.

5. Create the Plan: Define the “System Message” for the planner as the goal. This instructs the LLM about the behaviour we expect our agent to have. The goal should be straightforward and use similar terminology available in the tools' descriptions.

6. Execute the plan: Connects the details defined in step 4 to the plan and triggers the invocation step for the plan to start.

A Semantic Kernel plugin is a software component designed to extend the LLM's functionality by either granting the LLM capabilities that it did not previously have (code plugins) or defining the purpose of the LLM invocation to a specific topic (prompt plugins).

In contrast to tools supported natively by the LLM (such as function invocation), a plugin is not attached directly to the model but is placed within the planner. The planner will then generate a step-by-step approach and see if any of the steps of the goal are associated with the plugin and will call the function in question.

The main drawback of plugins is the requirement for a string response from the functions, as structured data is easier to manipulate in code and we don’t have to use resources ‘stringifying’ and parsing all our content. This is due to Semantic Kernel not being aware of the planner's plan before it is in action, making it so that all responses need to be compatible with an LLM call, which defaults to text.

Code plugins expand functionality by granting the LLM new “skills” it did not have access to prior.

The function is a standard Python function. To transform this to a code input we added the function decorator called kernel_function. This decorator expects 2 arguments:

• name, which is the name of the function to be referenced when attaching the plugin.

• description, which is a string value that indicates what the intention of the function is. This helps the planner identify when it needs to call this function and when it should avoid it.

The two main use cases for code plugins are the data based on the developer’s intention, or accessing external APIs. We demonstrate both abilities below:

Accessing an external API

As shown in the code snippet above, we want to retrieve the historical data for a specific ticker using the Yahoo Finance API.

We created the base function for it when we instantiate a Ticker based on the input. Notice that the model is able to convert the string input to the required variables, before calling the plugins.

Then, we use the historical data to get the closing prices for the period specified. Finally, we manipulate the pandas DataFrame to format the style of the columns Close and the date index as these will be used in future steps of the process.

Processing data

We want to calculate the drawdown for the data based on the formula: drawdown = current - cummax. Pandas provides us with the functionality required to perform this calculation.

Our original data looked like this:

After the function run our data looks like this:

Now that we have our data available, we will be moving to a prompt plugin to generate dynamic code.

Prompt plugins are used to alter the original intention of the LLM being invoked by providing it with new instructions. The plugin files are a prompt file and a JSON file. The prompt is the instructions that the LLM must abide by. The JSON file contains the LLM values to take into account.

The content of these files looks like this:

The keyword filenames skprompt and config will let the planner know that these are prompt plugins. The prompt file will contain a straight prompt instruction similar to an LLM Model system message, which provides instructions to the model.

The config file will determine the kwargs information for the prompt. This is the equivalent of writing the below code but with the correct mappings, and the input required based on the part of the goal this step is aiming to achieve:

The plugins are then attached to the kernel instance before we instantiate the planner. Hence, any planners we may have will benefit from the plugins provided.

Various tools are available to create GenAI agents for specific tasks, each with its strengths and weaknesses. Semantic Kernel is an interesting tool for creating straightforward agents that perform relatively well for simple tasks.

As illustrated above, we can use Semantic Kernel to quickly create AI agents that can retrieve and modify data as well as perform LLM requests to accomplish a predetermined goal. Semantic Kernel is capable of utilising the LLM to create the best approach to a task and which plugins to consume to ensure the task is completed.

The two main advantages explored in this article are Plugins and Planners. Plugins are versatile and powerful and require minimal alteration to functions for code plugins, and standard text values for prompt plugins. The variety of the planners and the ease of creation makes them very reliable for different types of agents.

This article explores the features of Maestro, a promising tool for mobile UI automation, and compares it with other popular tools such as Robot Framework Appium Library and WebdriverIO.

A common challenge in mobile UI testing is managing the complexity of test setups and ensuring consistent results across different environments. Maestro offers a solution by enabling users to perform tests using a simple YAML syntax, which can be easily understood and written by anyone in the organisation. This flexibility allows teams to quickly adapt to changes and make reliable decisions promptly.

Here is an example of how a Maestro test case is structured using YAML:

This example demonstrates a simple yet comprehensive test case for a hypothetical Wikipedia application:

1. appId: Specifies the ID of the application to be tested.

2. tapOn: "ADD OR EDIT.*": Simulates a tap on any element whose text matches the pattern "ADD OR EDIT".

3. tapOn: "ADD LANGUAGE": Simulates a tap on the "ADD LANGUAGE" button.

4. tapOn (with id): Simulates a tap on an element whose ID matches the pattern "menu_search_language".

5. inputText: "French": Inputs the text "French" into the input field.

6. assertVisible: "Français": Verifies that the text "Français" is visible on the screen.

7. tapOn: "Français": Simulates a tap on the "Français" option.

8. tapOn: "Back": Simulates a tap on the "Back" button to return to the previous screen.

Running tests in different languages can introduce complications, and this is true for any testing approach. With Maestro, this issue arises when the simulator's language settings do not match the language expected by the test scripts. For instance, if your simulator is set to Italian and your tests are designed for an English interface, the tests will fail because the locators (which depend on text strings) will not match.

To address this, Maestro provides the ability to set up and configure simulators for different locales. Here are some solutions:

• Locale configuration: Ensure that the simulator's locale matches the language of your test scripts. You can configure the locale using Maestro's command-line options:

• Use of IDs and universal locators: Instead of relying solely on text strings, use element IDs and universal locators that remain consistent across different languages.

• Parameterised tests: Create parameterised test scripts that can adapt to different locales by using variables for text strings, making it easier to run the same tests in multiple languages.

By implementing these strategies, you can minimise discrepancies and ensure more reliable test results across various language settings with Maestro.

Setting up emulators for different locales and operating system versions can be a challenge with Maestro. For instance, initiating a test run on an iOS simulator in British English requires specific commands and configurations:

It’s important to note that Maestro supports specific OS versions, such as iOS 15, 16, and 17, at the time of writing. For the most current list of supported versions, refer to the Maestro documentation. Setting up the appropriate emulator can be tricky, especially for those not familiar with mobile testing environments.

A critical aspect of mobile testing is the accurate inspection of elements within the application. Maestro Studio is a tool within Maestro that helps testers design and execute tests through a visual interface. However, relying solely on Maestro Studio may not always allow for precise element inspection, leading to flaky tests if elements are not correctly identified.

Element IDs are unique identifiers for UI elements within an app, similar to how IDs are used in web applications. These IDs are crucial for ensuring tests interact with the correct elements. Without access to the app’s codebase, it can be challenging to inspect and identify these elements accurately.

To address this, tools like Appium Inspector can be used alongside Maestro. Appium Inspector provides a comprehensive solution for inspecting app elements, allowing testers to accurately identify and interact with elements even without direct access to the codebase. A common practice is to use such inspectors to identify elements and reference the app’s codebase when possible to retrieve IDs.

By combining these tools, testers can ensure more reliable and accurate test results, reducing the likelihood of flaky tests.

One of Maestro’s main features is its integration of artificial intelligence (AI), which assists users in various ways:

• Automated element identification: The AI helps automatically identify UI elements, reducing the time needed to write test cases.

• Intelligent test recommendations: Based on the application’s structure and previous test cases, Maestro's AI can suggest test steps and improvements, ensuring more comprehensive test coverage.

Thanks to these AI capabilities, Maestro not only reduces setup time but also enhances the reliability of tests. Teams can then focus more on the core functionalities of their applications, ensuring higher quality and a faster time-to-market. This integration of AI into the testing process is relatively new, making Maestro an innovative tool in the mobile testing landscape.

While Maestro simplifies many aspects of mobile testing, it does not support running tests in parallel across different simulators directly. Even with two simulators and two terminals open, parallel execution is not possible. There is, however, the option to run tests in the cloud within a mobile development environment, although this incurs additional costs.

Choosing the right tool for mobile test automation can significantly impact efficiency, accessibility, and test maintenance. Here, we compare three main tools: Maestro, Robot Framework Appium Library, and WebdriverIO.

Maestro is designed for simplicity and ease of use, making it ideal for teams with limited coding experience. Its YAML-based syntax allows for quick test creation and execution, which is particularly useful for small projects or teams new to automation. The integration of AI in Maestro Studio provides intelligent test recommendations and automated element identification, enhancing productivity and test reliability.

Robot Framework Appium Library leverages the power of Python and the flexibility of Robot Framework. It is suited for teams with a solid coding background who need a robust and versatile testing solution. This tool offers comprehensive support for various mobile testing scenarios but requires more setup and a steeper learning curve compared to Maestro.

WebdriverIO is a JavaScript-based framework that supports both web and mobile testing. It is highly extensible and integrates well with various CI/CD pipelines, making it suitable for teams already familiar with JavaScript. WebdriverIO strikes a balance between usability and flexibility, offering powerful features for both beginner and advanced testers.

• Maestro: Recommended for teams seeking quick setup and ease of use, especially when team members have limited coding knowledge. Ideal for small to medium projects where speed and simplicity are crucial.

• Robot Framework Appium Library: Best for teams with strong coding skills who require a highly versatile and robust testing solution. Suitable for complex projects that demand extensive customisation.

• WebdriverIO: Great for teams with JavaScript expertise looking for a flexible tool that can handle both web and mobile testing. Perfect for projects that benefit from extensive integrations and a balance of ease of use and power.

By understanding the strengths and appropriate use cases for each tool, teams can make informed decisions to enhance their mobile testing strategy.

Based on our experience, we consider Maestro an exceptionally valuable tool for those new to mobile UI automation. Its simplicity in setup and use makes it particularly suitable for both small projects and those looking to gain a basic understanding of UI automation without facing a steep learning curve. However, for more complex projects, it might be necessary to opt for a more powerful and versatile tool.

Maestro serves as a precious resource for organisations aiming to improve the quality of their mobile applications through efficient and manageable testing processes. By reducing the complexity and time required to configure and execute tests, Maestro enables teams to focus more on innovation, reducing time to market and enhancing the quality of the final products.

In one of my recent projects, we encountered the need to migrate an application from a self-managed Kubernetes cluster hosted on AWS to Azure Kubernetes Services. This transition was not just a simple lift-and-shift, as it also involved enabling secure access to cloud resources across different cloud providers.

Official documentation often falls short in covering these complex scenarios comprehensively, leading to potential gaps in implementation. Through this blog post, I aim to share insights gained from implementing cross-cloud access, focusing on the integration between Azure and AWS. This discussion will cover two key scenarios:

1. Enabling access to AWS services from an Azure AKS cluster.

2. Facilitating access to Azure services from an AWS EKS cluster.

Our goal is to foster a harmonious interaction between these two cloud giants, overcoming the common obstacles encountered in such integrations and securely granting access without the need for sharing credentials.

Each cloud provider offers a bespoke solution to ensure secure access within their managed Kubernetes clusters. AWS employs the IRSA, allowing workloads to safely assume roles that provide necessary permissions to access cloud resources. Concurrently, Azure has rolled out Azure Workload Identity to streamline identity and access management. Both platforms require a specific add-on to inject tokens correlated with the service account.

Central to these mechanisms is OpenID Connect (OIDC), a robust and time-tested protocol foundational to identity verification across disparate platforms. OIDC is pivotal for securely interconnecting Azure and AWS services, offering a secure, scalable, and adaptable authentication framework that is crucial to our approach, guaranteeing smooth and dependable identity management across cloud environments.

OpenID Connect (OIDC) is a crucial protocol for verifying identities across various cloud platforms, enhancing the OAuth 2.0 framework by securely verifying a user's identity through an authorisation server and enabling access to resource servers. Although frequently used, the complexities of OAuth 2.0 and OIDC can initially be overwhelming, prompting a deeper exploration to understand how these protocols facilitate secure cross-cloud access.

In essence, accessing specific data or services requires authorisation to ensure the request is permitted and identity verification to confirm that the requestor is indeed who they claim to be. While OAuth 2.0 handles authorisation, it does not inherently confirm the requester's identity. OIDC addresses this gap by adding a layer of identity verification atop OAuth 2.0.

Core components involved:

• User Data: The data or services needed, managed by a resource server.

• Resource Owner: The owner of the data or service.

• Authorisation Server: Authorises access requests.

• Relying Party/Client Application: Initiates access requests. Upon approval from the Resource Owner, the Authorisation Server issues an "Access Token."

This framework ensures that data access is both authenticated and authorised.

The OAuth 2.0 protocol includes various flows to accommodate different scenarios:

1. Authorisation code flow: Used in systems like "Sign in with Google," where a user logs in to share specific data, receiving an access token in exchange for an authorisation code.

2. Implicit flow: Previously popular in applications that couldn't securely store secrets, such as single-page applications. Now, due to security concerns, more secure methods like the Authorisation Code Flow with PKCE are recommended.

3. Password credential flow: Employed when there is a strong trust relationship between the resource owner and the client, such as personal mobile apps.

4. Client credential flow: Ideal for server-to-server interactions, where a client application uses its credentials to obtain an access token directly, granting access to the resource server.

In all these flows, OIDC enhances the process by adding an "ID Token" to verify the user's identity, providing a robust identity layer over the existing OAuth 2.0 authorisation structure. For further details, you can refer to the official OpenID Connect documentation.

In this section, I share detailed authentication steps for AWS IRSA with Amazon EKS and Azure Workload Identity in AKS.

In AWS IRSA (IAM Roles for Service Accounts) within Amazon EKS, the OAuth flow used is akin to the Client Credentials Grant. This design is specifically tailored for machine-to-machine communication where an application acts on its own behalf rather than acting for a user. AWS IAM plays a critical role here, validating service account tokens issued by EKS and providing scoped AWS credentials for accessing AWS resources.

Detailed authentication steps in AWS IRSA using an OIDC provider:

1. OIDC provider configuration:

• Within AWS EKS, an OIDC provider is configured to connect the Kubernetes cluster's identity system with AWS IAM. This OIDC provider is not a separate service but a configuration within AWS IAM that trusts the Kubernetes cluster's service account tokens.

• The OIDC provider in AWS is set up to trust tokens issued by the Kubernetes cluster, meaning it recognises these tokens as authentic credentials.

2. Service account token:

• Each Kubernetes pod can be assigned a specific service account. This account has a JWT (JSON Web Token) associated with it, which is automatically managed and rotated by EKS.

• This JWT includes claims that identify the particular service account and permissions.

3. Authentication process:

• When a pod in EKS needs to access AWS resources, it retrieves its JWT from the file system (mounted by EKS into the pod).

• The pod or the application inside it then presents this JWT in a request to AWS IAM as part of the API call to assume the linked IAM role.

4. Token validation:

• AWS IAM, acting as the OIDC provider, validates the JWT against the issuer URL, the audience (aud), and other claims included in the JWT. These validations ensure that the token is indeed issued by the trusted EKS cluster and that it is intended for the specific IAM role it is requesting.

• This validation process is crucial as it confirms the authenticity and appropriateness of the request, effectively authenticating the service account's identity based on the OIDC standards.

5. Credential issuance:

• Upon successful validation, AWS IAM issues AWS credentials (access key, secret key, and session token) to the pod. These credentials are temporary and scoped to the permissions defined in the IAM role associated with the service account.

6. Using AWS resources:

• With these credentials, the pod can make authenticated and authorised API calls to AWS services, operating within the permissions boundaries set by the IAM role.

For more information, refer to the official documentation.

Similarly, in Azure Kubernetes Service (AKS), the Azure Workload Identity utilises a flow comparable to the OAuth 2.0 Client Credentials Grant. This setup allows Kubernetes pods to securely access Azure resources by leveraging Azure Active Directory (AAD) identities. The AKS manages the association between Kubernetes service accounts and AAD identities, facilitating a secure and scalable authentication method without user interaction.

Detailed authentication steps in Azure AD Workload Identity for Kubernetes:

1. OIDC provider configuration:

Azure AD (Active Directory) Workload Identity uses Azure Active Directory (AAD) as the OIDC provider. This configuration involves setting up Kubernetes to issue tokens that Azure AD can trust. The integration between Kubernetes and Azure AD allows Kubernetes service accounts to use these tokens to authenticate against Azure resources securely.

2. Service account token:

In Azure Kubernetes Service (AKS), each pod can use a Kubernetes service account that is automatically integrated with Azure AD using the Azure Workload Identity setup. The tokens issued to these service accounts are projected into the pod's file system and are used for authentication with Azure AD.

3. Authentication process:

• A pod needing to access Azure resources will use the token provided to its Kubernetes service account.

• This token is presented to Azure AD as part of the request to access Azure resources.

4. Token validation:

• Azure AD validates the Kubernetes service account token using established trust configurations. It checks the issuer, the audience, and other claims in the token to ensure it is valid and issued by a trusted Kubernetes cluster.

• This validation is crucial as it ensures the token's authenticity and appropriateness for the requested Azure resource access.

5. Credential issuance:

• Once the token is validated, Azure AD issues an Azure access token to the pod.

• This token is specifically scoped to the permissions that are assigned to the Azure AD identity linked with the Kubernetes service account.

6. Using Azure resources:

• The pod uses the Azure access token to make authenticated and authorised API calls to Azure services.

• These services validate the Azure access token and provide access based on the permissions configured for the Azure AD identity.

For more information, refer to the official documentation.

Following in the footsteps of IRSA in EKS, we simply need to substitute the EKS cluster with the AKS cluster. AKS hosts a public endpoint to expose the OpenID Configuration via the OIDC Issuer URL. Next, we need to create an IAM Identity Provider (IDP) that points to the AKS OIDC Issuer URL. This IDP is then associated with an IAM Role as a trust policy, along with a policy that grants the required permissions needed to access AWS services. Finally, we annotate the service account in the AKS cluster with the IAM Role ARN (Amazon Resource Name) that we created earlier.

In detail, we need to follow the below steps:

1. In Azure: Create a resource group that hosts the AKS cluster.

2. In Azure: Create an AKS cluster and enable the OIDC Issuer.

3. In Azure: Create a service account (SA)

4. In Azure: Get the OIDC Issuer URL.

5. In AWS: Create a resource (an S3 bucket) to check the access.

6. In AWS: Create a role providing access to the S3 bucket created above.

7. In AWS: Create an identity provider that points to the AKS OIDC Issuer URL.

8. In Azure: Annotate the SA in AKS with the IAM Role ARN that we created above.

9. In Azure: Install the amazon-eks-pod-identity helm chart.

10. In Azure: Create a workload that uses the SA and try accessing the AWS Service in this case S3 bucket.

In the next section, we’ll now get our hands dirty. I have used azure cli in the various steps you’re about to read — make sure that you have authenticated to Azure using your credentials.

At the end of the next section, our solution will look like this

Run the below commands and note that we set some env variables which will build up over time and will be used later down the line.

First, you need to create an Amazon S3 bucket. Replace your bucket name with your desired bucket name and your region with the AWS region you want to use. Copy a file into the S3 bucket for testing later. Note that we created a dummy text file named test-file.txt for testing the access.

Create an Identity provider from the AWS console with the Azure OIDC Issuer URL from the above SERVICEACCOUNTISSUER and audience as `sts.amazonaws.com`

You need three pieces of information:

1. OIDC Issuer Url from AKS.

2. Audience which is sts.amazonaws.com.

3. CA Root cert thumbprint.

The first two pieces of info exist already, so now we need to get the CA Root Cert thumbprint.

Use the below policy JSON and save it to a file locally as s3-policy.json to consume later. Note the bucket name matches the name of the S3 bucket we created earlier.

Run the below command to create the custom policy. Please make sure that you configure aws cli.

Use the templated JSON below to create a trust policy document and save it as trust-policy-template.json.

We have a templated trust policy file named trust-policy-template.json and using the envsubst CLI replace the the template values from env variables which is ready for us to consume.

Now this has generated a new file trust-policy.json with the required configuration.

You create the role using the AWS CLI by specifying the role name and the trust relationship policy document. Save your trust policy JSON to a file, for instance, trust-policy.json, and then run the below AWS CLI command

Once the role is created, you can attach permission policies to define what resources and actions the role can access. If you already have a policy ARN to attach, you can use:

This webhook is for mutating pods that will require AWS IAM access and it can be installed using the helm chart below:

https://artifacthub.io/packages/helm/jkroepke/amazon-eks-pod-identity-webhook

Note that cert-manager is a pre-requisite for this add-on and you can install this from the official documentation https://cert-manager.io/docs/installation/.

Note that we are deploying a pod using the service account created earlier in this setup. This pod utilises the Amazon aws-cli image to execute some CLI commands and test the access.

After deployment, proceed to test the access with the following commands:

The following sections cover the steps required for you to run the Azure Workload Identity in an AWS EKS cluster, from creating the AWS EKS cluster to validating access.

I am using eksctl CLI for provisioning the cluster and please setup your aws cli with the required credentials. In my case, I am loading a default profile on my machine.

To check that we are able to access the secret from the KeyVault, check the logs of the pod kv-read from above which would display the secret value. Note that the image used here uses the Microsoft Authentication Library (MSAL) go library to make use of the Workload Identity to access the KeyVault.

OIDC plays a crucial role in verifying identities across various cloud platforms, simplifying the complex process of integrating identity verification into OAuth 2.0. It allows for seamless, secure communication between cloud providers by ensuring that identities are authenticated properly before granting access to resources. This is particularly beneficial in scenarios where services from multiple cloud providers, like AWS and Azure, need to interoperate securely without sharing credentials directly.

Azure has abstracted much of the complexity involved in setting up the OIDC provider, offering a more user-friendly experience compared to AWS. Azure's Workload Identity integration with Azure Active Directory (AAD) provides a streamlined process where the identity management is tightly coupled with the Azure ecosystem, reducing manual configurations and potential errors.

AWS, while more complex in its setup, offers robust and flexible configurations through its IAM Roles for Service Accounts (IRSA) mechanism. This approach, although requiring a deeper understanding and more detailed configuration, allows for finely-tuned access control and extensive customization options, catering to advanced use cases and specific security requirements.

• Obtain the thumbprint for an OpenID Connect identity provider

• Create an OpenID Connect provider on Azure Kubernetes Service (AKS)

• Server Certificate

• Diving into IAM Roles for Service Accounts

In our previous article, we discussed how to replicate data from a Postgres database to a Node.js application in real-time using logical replication. However, if the Node.js application crashes or stops for some reason, the replication will cease, and we risk losing the data that our system produces in the meantime via another microservice or application.

In this article, I discuss how to resume replication from the last point where the Node.js application stopped by using a persistent replication slot in the Postgres database. This ensures that our application doesn't lose events produced by other microservices or applications during downtime.

Editor’s note: This is a cross-post written by Senior Software Developer, Manuel Spigolon. Manuel has his own blog at backend.cafe where you can subscribe for updates and find more great posts. Some of the links in the article point to Manuel’s personal GitHub account.

To resume replication, we need to create a replication slot in the Postgres database. A replication slot is a logical entity that keeps track of changes happening in the database and sends them to the subscriber. The postgres package we used in the previous article automatically created a replication slot for us, but it was not persistent, it was a `TEMPORARY` replication slot that was removed when the subscriber disconnected.

Since we want to resume replication from the last point where the Node.js application was stopped, we need to create a persistent replication slot. Let's create one in a new setup-replication.js file:

We are using the `pgreplicationslots` view to check whether a replication slot with the given name already exists. If it doesn't exist then we create a new replication slot using the `pgcreatelogicalreplicationslot` function.

Note that we specified the `pgoutput` plugin in the function to decode the changes in the replication slot. This is the default plugin for logical replication, and it ships with Postgres. Be aware that there are other plugins, such as:

• `test_decoding` plugin is the simplest plugin that ships with Postgres to start building your own custom plugin.

• `wal2json`, which must be installed separately in the Postgres database, allowing you to use them in the pgcreatelogicalreplicationslot function.

Note that each plugin has its own advantages and disadvantages, so choose the one that best fits your use case. The biggest difference if you try to use test_decoding versus pgoutput is that the former does not accept a publication name as a parameter while the latter does. This means that you can use pgoutput to filter the changes you want to replicate, while test_decoding will replicate all changes in the database without filtering them!

Now, run the setup-replication.js file to create a replication slot!

In the previous article, we already created the setup-consumer.js that creates the publications that our application is interested in. So, we can reuse the same file and just run it — if we haven't already.

As a reminder: you will first need to start the Postgres server and create the foo database.

We are ready to create a new consumer-resume.js file that will resume replication from the last point where the Node.js application was stopped, so let's jump into it:

This time, we are using the `pg-logical-replication package` to demonstrate the resuming of replication. Its low-level API provides us more control over the replication process, otherwise we would not be able to configure the Plugin to receive only the changes we are interested in.

The code can be explained as follows:

• 1: We are creating a new LogicalReplicationService instance and passing the connection options to it. Note that we are setting the acknowledge.auto option to false to manually acknowledge the changes; otherwise, they would be automatically acknowledged. By setting this option to false, we gain even more control over the process.

• 2: We are listening to the data event to receive changes from the replication slot.

  • At this point, you should process the log and apply your business logic. In this case, we're just logging the changes to the console.

  • After processing the changes, you must acknowledge them using the acknowledge method; otherwise, the slot will not advance. The lsn (Log Sequence Number) is the unique identifier for each change in the database and is used to track changes in the replication slot.

• 3: We are creating a new PgoutputPlugin instance and passing it to the subscribe method to establish a connection with the replication slot.

To start the application, run the node consumer-resume.js file, and it will begin receiving changes from the replication slot. If we did all the steps correctly, you can start the node producer.js file that we wrote in the previous article to produce changes in the database and see the changes in the consumer application.

If you stop the consumer application by pressing Ctrl+C, the replication will stop, and the slot will not move forward. However, if you start the consumer-resume.js application again, it will resume replication from the last point where it was stopped! 🎉

Moreover, we can see that the output shows only the changes from the foo_odd and fooupdateonly publications, which we configured in the PgoutputPlugin instance so we will see updates and inserts with odd id numbers only:

In this article, we discussed how to resume replication from the last point where the Node.js application was stopped.

We created a persistent replication slot in the Postgres database and used the pg-logical-replication package to demonstrate resuming replication. This ensures that our application doesn't lose data produced by other microservices or applications during downtime.

In doing so, we did not change the producer.js file, which means that the producer can continue to produce changes in the database without any issues and the previous Publications setup is still valid: we just configured manually the replication slot and the new consumer.

Remember, the replication slot retains changes in the database until the slot is dropped or the changes are acknowledged by the subscriber. If not managed properly, this can lead to high disk usage because Postgres will keep the changes in the WAL logs indefinitely instead of removing them.

I hope you enjoyed this article and learned something new! Does it deserve a comment and a share? 🚀

True, to a point, but in reality, it’s not that these companies aren’t integrating cloud services, or that they’re dragging their feet in doing so. They’re just not fully embracing them — yet. 

Though caution is understandable, there is some urgency for leaders to make the switch. McKinsey reports that “Fortune 500 financial institutions alone could generate as much as $60 billion to $80 billion in run-rate EBITDA in 2030 by making the most of the cost-optimization levers and business use cases unlocked by cloud.”

Financial services executives recognise that in order to compete and stand out in an increasingly crowded marketplace, the speed, power, and flexibility afforded by cloud computing is critical. Most financial services companies are indeed exploring how moving to the cloud can benefit their businesses, but many are still only scratching the surface of what’s possible. 

Graphic: GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Considering investments in legacy systems and concerns about regulatory compliance and data security, hesitancy to go all-in on the cloud has been understandable. However, changing regulations and the  obligation for financial services organisations to have the resilience to identify potential disruptions and minimise their impact highlights the importance of modernising with cloud-native systems.

Threats to business continuity encompass human-made attacks such as cybersecurity breaches, unpredictable events like power outages or natural disasters and potentially preventable issues including hardware or software failure. Digitally resilient organisations analyse available information to anticipate potential disruptions, and have the resources to minimise and recover from the impact of disruptions that do occur. For financial services organisations in particular, disruptions can mean not only lost revenue for the affected company, but also losses for customers and negative effects to the larger economy. For this reason, digital resilience is even more necessary. 

This necessity extends beyond simply responsible business practices - new regulations, including the Digital Operational Resilience Act (DORA) in the European Union, require financial services companies to demonstrate that they have created thorough incident response plans and infrastructure in place to prepare for and mitigate interruptions of service. According to the Act, responsibility to comply lies not just with financial companies as a whole, but with individual officers of the company. The text states “Under DORA, the Board of Directors is personally liable for cybersecurity governance and risk management, including all aspects such as reporting, testing and other necessary measures.” 

In the United States, the Federal Reserve Board, Office of the Comptroller of the Currency, and the Federal Deposit Insurance Corporation worked together to develop and issue the “Sound Practices to Strengthen Operational Resilience” guidance. This guidance explicitly defines practices that large banks should undertake to prepare for and address the operational risk of cyberattacks, natural disasters, and pandemics. A similar group in the UK, comprised of the UK's supervisory authorities, the Prudential Regulation Authority (PRA), Financial Conduct Authority (FCA) and Bank of England (BoE) announced resiliency requirements for UK banks and insurers. The purpose of all of these coalitions is to ensure that the financial institutions operating in their countries are taking appropriate and adequate steps to protect consumers, the overall financial sector, and country economies. 

The prospect of complying with regulations can sound daunting, but in the case of regulations instituted to promote digital resilience, there are actual economic benefits. Key requirements of DORA and other operational resilience guidance documents are incident-response plans, business continuity plans, and regular risk assessments. Regulations aside, all of these assets are important for companies to have in place and up-to-date in order to limit financial losses in the event of a disruption. Fees for noncompliance with regulations can be steep, but they would be minimal in comparison to the damage done by a major event that an organisation is unprepared for.

It makes financial sense for companies to invest in resilience, and many are in the process of doing so. Studies show that executives in financial services organisations are decreasing spending on legacy systems, and increasing it on technologies that boost productivity and promote resilience such as cloud services. In 2022, 53% of banking tech executives surveyed expressed that they planned to increase investment in cloud platforms by the largest amount compared to other tech spending**. McKinsey reports that by 2030, value drivers could enable cloud services to deliver more than $3 trillion in EBITDA value, $407 billion of it in IT resilience improvement, across the Forbes Global 2000.

**Data source: Tool: Cloud Computing Use Cases for Banking and Investment Services, 2023, 21 June 2023 - ID G00796050 GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Cloud services are a key part of building resilience due to their ability to help organisations prevent, respond, and recover from disruption. They’re built to be reliable, with redundant systems, backup mechanisms and disaster recovery plans in place. These help minimise the risk of service interruptions and ensure applications and data are accessible even in the event of hardware failures or other disruptions. 

Additionally, cloud service providers handle the technical processes of database management and security. They also monitor and analyse their systems, which creates the potential to identify and address potential service delivery issues before they escalate, helping to maintain high levels of service availability and performance. Managed service plans enable organisations to contract these operational tasks to a third-party vendor, and focus their resources on application development and improvement.

When it becomes necessary to respond to an event, Cloud Service Provider (CSP) flexibility enables quick adjustments to resources and configurations in response to service delivery issues such as performance bottlenecks or hardware failures. 

In the recovery phase, cloud-based systems are able to recover from issues faster than traditional on-premises infrastructure. CSPs handle underlying infrastructure management, allowing organisations to focus on restoring service rather than troubleshooting hardware or software issues. Additionally, automated backups and data replication enables companies to quickly recover their data and applications if a truly disastrous event wipes out the existing infrastructure.

The reasons holding many financial services companies back from migrating more of their business to the cloud are fading away. But the complexity of modernising legacy systems or integrating them with cloud-based solutions requires an experienced partner who truly understands an organisation’s business and technology needs.

In its work with IBM, Nearform leveraged its cloud expertise to develop a leading-edge, open banking app that enables a complex, real-world customer journey.

When diagnosing how to help a client’s business be more resilient, Nearform uses a proprietary method  to get an overview of a business’ structure and operations. During the first phase, the goal is to get a holistic understanding of system issues and where problems may lay, and objectively assess the client’s technology, resources, and processes in place.

In this case study, Nearform identified core issues with monitoring and alerting, incident management and observability. The diagnostic phase revealed an over-reliance on manual operations, and an inability to effectively prioritise tasks, making the team reactive instead of proactive. These insights led the team to the solutions, and provided a clear path to improve the company’s resiliency.

There’s no denying that cloud migration is a complicated and challenging process. With an experienced partner to help identify the most effective path forward and design a secure, resilient customised digital solution, financial services companies can fully embrace the power and versatility of the cloud. 

To combat the growing complexity and bolster compliance efforts, emerging technologies like Artificial Intelligence (AI) have proven to be helpful. AI shows promise in helping to manage complicated and changing requirements, automating repetitive tasks, as well as giving human representatives more time to focus on strategic compliance efforts.

* Data source: Financial Services Business Priority Tracker 4Q23 2024 Gartner, Inc. and/or its affiliates. All rights reserved. 807706_C GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Due to increased regulatory requirements and heightened attention to compliance, many financial services companies are increasing their focus on, and investment in, ensuring that they’re compliant. Estimates show that financial institutions will increase their spending on regulatory technology (RegTech) investments by approximately 124% between 2023 and 2028.

Driving this additional investment is the increase in fines for noncompliance with record keeping, and other regulations. In a survey, 69% of financial services executives reported that they expect regulators to increase the value of fines for record-keeping breaches.

Graphic: 2024 THE STATE OF FINANCIAL SERVICES COMPLIANCE Annual Compliance Health Check Report

In February 2024 alone, the SEC imposed $81 million in fines for record-keeping infractions. Globally, fines have been assessed to companies that failed to record or retain electronic communications information. Some of the largest institutions including Wells Fargo, JP Morgan, Goldman Sachs, Morgan Stanley and Citigroup have been punished with fines in the hundreds of millions, with total penalties exceeding $2 Billion. 

Factors contributing to the increasing number of regulations and strict enforcement include global geopolitical conflicts and changes, technological advancements in financial industry systems, and the use of artificial intelligence in financial crime.

Changing circumstances in global markets, developments in technology, and organisation-specific strategies can affect which of the many areas of compliance pose the greatest risk, and therefore require the greatest attention from financial service company leaders.

Some of the current areas that many organisations are focusing on are: Artificial Intelligence (AI), operational resilience, compliance risk assessment, compliance monitoring, and environmental, social and governance (ESG issues).

Compliance and governance are typically already part of most digital systems, and there are a number of different procedures, checks, and monitoring capabilities that assist compliance efforts. These include regulatory compliance checks such as Anti-Money Laundering (AML), Know Your Customer (KYC), General Data Protection Regulation (GDPR), and others. Data encryption and security measures are required to keep customer data secure, and records of all user activities and transactions (audit trails) are required in order to show that organisations are transparent and accountable. 

Beyond the digital applications and modules that support compliance, there are also important actions that employees and staff are required to undertake. These include participation in compliance training and awareness programs, and conducting regular audits and assessments. Well-designed systems and thorough, informed oversight by human auditors can keep organisations from falling out of compliance, but the increasing complexity of regulatory requirements makes it more challenging to keep up with all relevant regulations.

GenAI (generative AI) is proving to be a valuable asset in the area of regulatory compliance. When trained with information about regulations and policies, it can serve as a virtual “expert”, helping to assess compliance by comparing required policies and regulations with company policies, and answering user questions about regulations. Developers are using it as a code accelerator as well, checking code for any misalignment with policy. When it detects possible breaches, it can alert the necessary people, and detail the situation clearly.

Additionally, AI can be very good at detecting and mitigating cyberfraud. When trained with lots of data showing what legitimate and fraudulent transactions look like, AI systems can monitor activity, then identify and block potentially fraudulent transactions. It can also communicate with customers to ask for more information or confirm details, which minimises the threat from identity theft, phishing attacks, credit card theft, and document forgery.

Task automation is another area where AI excels. It speeds up completion of repetitive processes like document verification, data analysis, transaction monitoring and more, and eliminates mistakes caused by human error.

The promise of AI to help support compliance is counterbalanced by concerns about vulnerabilities that it may be used to exploit. AI has the potential to cause harm by facilitating cybercrimes, impersonating humans, and presenting incorrect information as correct. Concerns such as these are driving many governments around the world to consider instituting regulations to govern it. These regulations and national policies are evolving as quickly as the technology itself, so AI-specific updates to existing rules and new additions will soon be in effect.

Adding to the complexity for international organisations, regulations will differ by country, and sometimes, even within countries or regions. 

For example, while AI policy is still in development in the US, indications are that different agencies will have the ability to create their own rules. Having several sets of rules to follow from different agencies or even from different states will add levels of complexity to compliance efforts.

The UK is seeking to be pro-innovation, encouraging AI development and setting broad policy guidelines while emphasising five cross-sectoral principles for existing regulators to interpret and apply. Those principles are: safety, security and robustness, appropriate transparency and explainability, fairness, accountability and governance, and contestability and redress. Since regulators are still in the process of creating a cohesive policy, there’s no specific timeline for when regulations will be released, 

While AI policy is still under development in these countries and others like China, Australia, and Brazil, The EU has produced the world’s first comprehensive law governing AI, which will come into effect in 2024. This law will ban some uses of AI, require companies to be transparent about how they develop and train their models, and hold them accountable for any harm. It will also require AI-generated content to be labelled, and create a system for citizens to lodge complaints if they believe they have been harmed by an AI system.

First and foremost, responsible AI development means ensuring that people are guiding and reviewing the results of AI models. There’s no substitute for human verification that models are responding accurately, giving correct results, and that the data is correct and unbiased.

In addition to keeping humans in the loop, McKinsey outlines some steps that financial organisations should take to manage the risks of GenAI: 

• Ensure that everyone across the organisation is aware of the risks inherent in GenAI, publishing dos and don’ts and setting risk guardrails.

• Update model identification criteria and model risk policy (in line with regulations such as the EU AI Act) to enable the identification and classification of GenAI models, and have an appropriate risk assessment and control framework in place.

• Develop GenAI risk and compliance experts who can work directly with frontline development teams on new products and customer journeys.

• Revisit existing know-your-customer, anti-money laundering, fraud, and cyber controls to ensure that they are still effective in a GenAI-enabled world.

Despite valid concerns, Artificial Intelligence can empower financial services companies to more effectively comply with changing regulations. The concerns should not be ignored, but with responsible and transparent development, they can be neutralised. 

As AI continues to grow in importance, overly cautious adoption may lead to competitive disadvantages, while overly rapid implementation could introduce serious compliance risks. Partnering with an experienced AI and data consultancy such as Nearform can ensure that organisations take a balanced approach that is tailored to their needs.

With a proven track record of working with major financial services organisations and global enterprises, Nearform helps unlock the value of AI safely and strategically, positioning companies for future success in a rapidly evolving landscape.

When it comes to API testing, Postman has been the tool of choice for countless organisations. Utilised for its easy-to-use interface, Postman simplifies the API testing process and allows you to incorporate response validation tests using JavaScript, enabling users to ensure the accuracy and reliability of API responses.

Another big advantage of Postman is its flexibility when using different environments. Whether you’re testing locally, on staging servers, or in production environments, Postman provides a streamlined user experience, allowing users to adapt their testing strategies with ease.

In one of our projects we used GraphQL queries tailored to each environment, and we needed to ensure thorough testing across different setups. We implemented an HTML report feature, offering a visually engaging summary of our findings.

This report not only facilitated clear communication of our test results to our stakeholders, but also aided in identifying any discrepancies or areas for improvement. We were also able to streamline our testing process by integrating our collection into the CI/CD pipeline using Newman. This integration allowed for nightly executions of the collection without encountering any operational issues, enabling continuous monitoring of the services' performance and stability.

In late 2023, Postman announced it was making some significant modifications to its pricing and functionality structure. We were using the free desktop version, now known as the lightweight API, which became impractical for our needs.

While the prospect of upgrading to the enterprise version to access the necessary functionalities seemed plausible, it unveiled a dealbreaker for our client: data security.

Transitioning to the enterprise version mandated data synchronisation with the cloud, a feature that clashed with our client's security protocols. Further compounding this challenge was the absence of an offline version in Postman's offerings, except for the lightweight API. Consequently, we had to identify a suitable alternative tool that could accommodate our testing needs while adhering to our client's security standards.

Bruno is an alternative to Postman which offers some key features that immediately appeared useful for our migration activities:

1. User interface

2. Importing collections

3. Assertions

4. Visual Studio Code extension

5. Secrets

Bruno’s UI is very similar to Postman’s. If you’re used to using Postman, you’ll know straight away where everything is. Postman’s UI was working; it flows, so it seems like Bruno was sticking to that flow.

In Bruno's 'Settings' section, you'll discover a feature that allows you to include scripts for acquiring access tokens or executing other prerequisite actions. Additionally, you can select the authentication mode for the entire collection. Interestingly, this functionality now appears to be absent from Postman's free version, marking a notable advantage for Bruno users.

When it comes to the query section, Bruno closely resembles Postman, with one notable addition: the 'Assertion' section. This extra component provides users with a dedicated space to define assertions, allowing for more comprehensive and efficient testing. We'll delve deeper into this feature later on.

The ability to import existing collections from Postman proved invaluable, particularly given our prior investment in Postman. Using this functionality, we migrated our extensive collection from Postman to Bruno.

However, it's worth noting that some adjustments were necessary during the migration process. Postman’s folder section of pre-script did not get imported, so we had to rewrite it, which we needed to acquire access tokens for our APIs. Additionally, Bruno employs its own set of keywords and conventions, necessitating updates to align our tests to keep them working.

Nevertheless, despite these minor modifications, the migration process was smooth and efficient, enabling us to swiftly transition our testing infrastructure to Bruno's platform without any impact on the validity or functionality of our test suites.

As a test engineer, I loved this feature that Postman does not have. It allows users to effortlessly build tests one after another, simplifying the testing process significantly.

Despite my appreciation for this feature, its utilisation was not necessary for our project at the time. Given that we already had an extensive collection established in Postman, our approach involved importing this existing collection into Bruno and subsequently updating it to align with Bruno.

The screenshot below shows Postman’s tests that we imported into Bruno. As you can see, you cannot use them as they are. We updated all our tests using grep which took only a small amount of time to make them executable again.

Postman offered an 'export' function, enabling users to work with exported Postman collections in Visual Studio Code. However, the current version has removed this functionality, making it impossible to share collections through Visual Studio Code.

Bruno comes with Visual Studio Extension. What truly set this feature apart was its synchronisation with the UI. Any modifications made within the UI were automatically reflected in the corresponding files. All the essential components, header data, request body, variables, and tests, were neatly organised in one file.

Within Bruno’s configuration settings, users have the capability to hide the values of variables designated as sensitive.

This functionality ensures that these values remain safeguarded within the environment settings. Consequently, they are accessible for local usage while being shielded from inadvertent exposure when pushing changes to remote repositories. The ability to hide secret values in this manner provides an added layer of security and compliance, aligning with industry best practices for safeguarding sensitive data.

The short answer is yes; it did for our project. Bruno was able to do everything Postman did.

The longer answer is that during the initial phases of our migration, we encountered a hurdle in extracting environment variables within our CI/CD pipeline. This posed a significant challenge, as integration with our automated pipeline was crucial for maintaining testing efficiency and workflow continuity.

Fortunately, Bruno's status as a relatively new tool with an engaged community proved to be advantageous. The platform undergoes frequent updates, driven by user feedback and community collaboration. As luck would have it, one of the recent updates addressed the issue we encountered, providing the solution needed to proceed with migrating our Postman collections to Bruno.

In addition, Bruno's user interface proved to be intuitive and user-friendly, particularly for those accustomed to Postman. The integration with a VS Code Extension enabled collaboration and sharing among team members, encouraging more engineers to actively contribute to the collection.

Previously, the responsibility for updating the collection primarily fell on test engineers in our project. However, post-migration developers became more involved in updating the collections alongside their application code changes, as both files resided in the same repository. This shift fostered greater collaboration between developers and test engineers, resulting in a smoother and more efficient testing process.

Following the migration, our API tests continued to run without issues. The increased collaboration between developers and test engineers contributed to the overall success of the migration. Encouraged by the smooth migration and the continuous evolution of Bruno, we recommend you give Bruno a try.

With a commitment to promoting learning within the developer community, Nearform has curated an impressive array of workshops covering a diverse range of topics, from backend frameworks like Fastify to security essentials like OWASP Top Ten and everything in between.

Each workshop is designed to combine theoretical knowledge with hands-on practice, ensuring participants gain a comprehensive understanding of the topic at hand. Participants are actively engaged in writing code and running tests to validate their understanding and mastery of the concepts covered.

With solutions readily available, participants can confidently solve each exercise knowing that assistance is just a step away. Let's take a closer look at some of the workshops.

Fastify, a web framework for Node.js, boasts lightning-fast performance and a minimalist design. Nearform's Fastify Workshop provides a comprehensive introduction to building web applications with Fastify. Whether you're a beginner or an experienced developer, this workshop offers valuable insights and hands-on exercises to master Fastify's features and best practices.

Here is an example of an exercise you can find in the workshop:

If need some help, you can find the solution in the source code:

And then you can test it with Postman or Curl:

Security is paramount in today's digital landscape, and Nearform's OWASP Top Ten Workshop equips developers with the essential knowledge to identify and mitigate common security vulnerabilities. By exploring the OWASP Top Ten risks, participants learn how to secure their applications and protect against threats effectively.

For example, in this exercise you have to fix a snippet of code with a SQL Injection vulnerability:

Let’s fix it by sanitising the user input:

To make sure you have fixed the vulnerability correctly, you can run npm run verify , which will test that your solutions address the security issue in the code.

GraphQL has revolutionised the way developers design and query APIs, offering flexibility and efficiency unmatched by traditional RESTful approaches. Nearform's GraphQL Workshop demystifies this powerful technology, guiding developers through the fundamentals of schema design, querying, and optimisation techniques.

Loaders are an amazing feature of GraphQL, in this exercise you will learn how to use them correctly:

And then you can check the solution and run the test:

Testing is an integral part of the software development process, ensuring the reliability and robustness of your applications.

Node.js released an experimental test runner in version 18 and made that test runner stable in version 20, allowing developers to test their applications without the need for external dependencies.

Nearform's Node Test Runner Workshop introduces developers to various testing methodologies and tools, empowering them to write comprehensive test suites and automate testing workflows effectively.

In this slide we explain the difference between a Test Runner and a Testing Framework:

Micro Frontends architecture enables teams to independently develop, deploy, and scale frontend components, fostering agility and collaboration in large-scale projects. Nearform's Micro Frontends Workshop guides developers through the principles and implementation strategies of micro frontend architecture, equipping them with the knowledge to build modular and scalable frontend systems.

With the rise of mobile app development, React Native has emerged as a popular framework for building cross-platform applications with JavaScript and React. Nearform's React Native Workshop empowers developers to harness the full potential of React Native, from setting up development environments to building native-quality mobile experiences.

Whether you're diving into backend development, exploring emerging technologies, or fortifying your application's security, Nearform's open-source workshops provide invaluable resources and guidance. Immerse yourself in these workshops, join a vibrant developer community and commence a journey of perpetual learning and advancement.

Here you can find the list to all our workshops.

So, which workshop will you venture into first? Let's embark on this learning journey together!

I am a strategy consultant-turned-marketer. I started my career at companies like The Parthenon Group (now Parthenon-EY) and McKinsey & Company — where I focused my time on the tech & startup industry, doing marketing strategy. 

I shifted gears to working in tech about 15 years ago, first doing mobile user acquisition as VP of  Marketing & Analytics at CoffeeTable, an e-commerce startup in San Francisco, and later shifting gears to do B2B marketing, with leadership roles at a Turkish fintech and later at McKinsey Digital Labs. 

I was thrilled to join Nearform in the summer of 2023 as Chief Marketing Officer.

Speaking at the Dublin Tech Summit was an amazing experience. It was great to bring together diverse perspectives on timely topics. I was lucky enough to speak on a panel with leaders from NASA, Microsoft and many other leading organisations from Europe and around the world.

Something we agreed on is the fact that digital transformation, as we know it or have known it in recent years, is fundamentally about to change — and a big driver of that is the advent of AI.

AI is going to change the way we do everything. It's going to change how companies like Nearform develop software, products and platforms, and how we work with data. It's also going to change how we serve clients, who include enterprises from sectors such as financial services, telco, healthcare and more. 

From a marketing perspective, the way customers receive services and get value from companies, and the way brands communicate, all of this is about to change with how AI is incorporated. A lot of that brings good news. You can see the potential for efficiency. You can see the potential for customers to get served with what they need in a faster way and, potentially, a better, more accurate way. You'll see efficiency gains in terms of production and the supply side. 

But with it are also inherent risks. There are concerns about data, privacy and security. There are certainly concerns around regulation, which currently is a patchwork of different approaches across different geographies. For example, the EU is about to pass laws that are very different from what the US is passing federally, which is complemented by what specific US states are passing separately. 

So, while it's an exciting area, and one that will fundamentally change how enterprises engage in business transformation, it's also one to really be vigilant about and have an approach that integrates risk mitigation along with execution in parallel.

Each of us brought different perspectives to the Dublin Tech Summit.

One of the speakers was a digital leader from NASA who has been there for 25 years. It was interesting to hear about the similarities, as well as differences, in the issues they face. 

A lot of the emphasis was on the importance of having a business wrapper around technological initiatives. This is key because there is a risk of developing technology for technology's sake — creating new features that weren't available before, but now can be delivered, simply because they're feasible. But in the absence of a business lens, this raises the risk of creating features that won't be fundamentally adopted by the market, by users, by consumers, by employees etc. 

I thought he brought a really interesting perspective in terms of that requirement, of the intersection of business and technology, one that aligns with Nearform’s approach of collaborating with both digital and business leaders across functions.

Another one of the speakers made a point about the importance of people in the transformation process. We talk about AI as a technology and as a digital initiative. But, by and large, it's actually a people project, because so much of what AI enables and or requires is change in processes. It will change the skills required to complete certain tasks and fulfil certain roles, but it also will require a redesign of processes and ways of working.

All of that requires behaviour change. As anyone who has worked on transformation initiatives with companies knows, a big goal is to change people's existing habits — break old habits and create new habits. 

I think the intersection of those two perspectives was really interesting.

The previous age was very much focused on thinking of digital transformation as a separate but parallel initiative to what the business objectives and business strategy are. 

Organisationally, CIOs, CTOs and CDOs were often very siloed from the business units themselves. So tech decisions would be seen as large investment decisions, and large capital outlays, the impact of which would oftentimes be measured at the business unit level. Sometimes at the corporate level, in the case of corporate strategy teams, and so forth. But they were seen as disparate streams of work.

Now business and technology are increasingly intertwined, so the key is to leverage today's technology in service of your business objectives. That means organisations are increasingly on the front foot of technology tools. Put another way, transformation is now multiple layers.

Digital transformation cannot be digital alone. It's almost a misnomer. The real challenge of any transformation effort is that it requires people to adapt their behaviour. If you underestimate the “people” aspect, both in terms of buy-in for the adoption of new tools as well as the capability building required to really leverage and get the most out of the investment and new tools, you won't be successful — no matter how great that shiny new product or platform may be.

There are a number of key technologies. Obviously data, and AI is the one that’s top of mind for everyone. But what AI means is different for each organisation, and it's a moving target.

At Nearform, we tell our clients we meet you where you are. This is because there are many factors that influence where an organisation sits on the data and on the AI maturity curves. The reason we say this is AI is not one point in time, it's not even a linear goal. It's generally the culmination of a multi-step process that's required. 

This starts from data engineering, just getting the plumbing working and having the data “pipes” around different parts of the organisation talking to each other. It then goes to enabling the data science required to unlock the analytics that can drive business intelligence, and ultimately insights and actionable decisions you can take.

That moves up to AI, where the human is no longer necessarily the decision maker. But the technology can take on that role. And all along the way are other critical areas, like data governance, reliability, security and much more.

Again, it's this notion of technology, capability and strategy all coming together as one.

Another consideration is: “Who is the end user of AI?” There are so many applications, and one that always comes to mind is the notion of chatbots and how they can serve customers. But you also have to think about internal applications of AI — for employees. There's an urgent need to reevaluate how work is done in the context of AI.

At its root, AI is a user experience strategy. It requires getting close to and understanding use cases and desired impacts — as opposed to technology for technology's sake or creating features for feature's sake. It's taking a holistic view of how that particular user is incentivised and is trained to operate.

There's increasingly this competitive dynamic between enterprises and newly emerging challenger brands. We see a lot of this in the financial services industry, with new branchless banks for example. We see that in numerous other industries as well. 

Investment can and should be made as close to the user as possible. As I mentioned earlier, individual business units within an organisation are increasingly involved in investment decisions around technology. This enables existing enterprises to reinvent themselves at a more rapid pace than was previously expected. Plus, they have the added advantage of having legacy knowledge and expertise, as well as a track record of success. 

While emerging technology may be seen as democratising access to markets for new brands, it can also provide new competitive advantages for existing brands too.

Fear of change is a natural human response, but I would say no in the short term. In fact, a lot of the initial disruption we're seeing is in the most frustrating or annoying — and least sexy — areas. 

For instance, I angel invest in American and European startups. I remember one of my first investments some years ago was for a company that helps digitise the note-taking process and elderly caretaker settings. The environment and the processes were not exactly cutting-edge. However, the user insights and the data that emerged helped develop a tool that led to a step function change. This change wasn’t just in the note-taking process, but also the employee engagement, as well as the patient care level.

I think the same is true today — when thinking about where to embed new technology in the day-to-day, that's different. You look first at: “What is it that people complain about?” It could be HR processes. (In my case, it's expense processes!)

There are different ways in which people can embrace new technologies, specifically AI-powered efforts that aren't scary and are actually really helpful in the day-to-day. For example, in my daily work, one of my colleagues on the Digital Marketing Team collaborated with one of our AI experts to personalise and automate a subset of our outreach messages. 

Something like that is a tremendous step forward in terms of providing the right messaging, at the right time, to the right person to help make our efforts more effective and helpful.

However, there do need to be clear steps taken towards risk mitigation. Again, that's the notion of data governance, regulatory compliance, cybersecurity etc. Those are all steps that need to be taken before and/or in parallel with embracing these new technology uses to ensure there's no need to be afraid, either now or in the future.

At Nearform, we say we meet you where you are. There are numerous factors that influence where an organisation sits on the data and AI maturity curve, so we use our experience to develop impactful solutions along that entire process.

For instance, we recently worked with a utilities company in South America. Nearform used AI to optimise the chemical facility operation based at a remote location to provide data that was expected to come from the facility even before it was operational. Our team built a digital twin of the physical facility, which enabled the new tech to go live before the facility launched. Within weeks, the Nearform team defined an end-to-end data flow, launched the platform and produced a functional prototype. And we followed that by delivering the full solution.

What I love about that example is it combines the best of what Nearform does. We do product, platform, data and AI as well as capability building to ensure our clients have a resolution at the end of an engagement that enables them and sets them up for success, moving forward after we're gone.

One of the things that really differentiates Nearform is our track record of success. We've been around since 2011, serving the biggest enterprises, governments and nonprofit organisations all over the world. We've improved the quality of patient care. We've helped governments get societies out of the COVID-19 pandemic, through data and through COVID tracking and so forth. We help airlines with their reservation system, so people don't get booted off of flights and things like that.

A lot of that has to do with the fact we only hire senior engineers, so our teams can get started and are able to get to results much faster than the average tech team. 

We also have a legacy of Open Source contributions. Nearform is one of the largest Open Source contributors in the world, and we're able to leverage that knowledge and those resources and tools to our client efforts. It’s another accelerant for getting time to value as quickly as possible.

Something else most people don't talk about is the fact that Nearform has a team most companies want to work with. People tell us: “Not only is your developer team sharp, not only are your designers and product managers collaborative, but they're really nice too .”

We bring a culture of shared success, and one of being on the same team, with our clients, as opposed to being seen as an outside vendor. We are truly a partner to our clients, one that’s on the same side of the room, whiteboarding together, working together, and having a human connection, in addition to the technology we deliver.

I've been a woman working in tech for many years, and I've worked in Silicon Valley as well as across the US and throughout Europe. I've seen or been in many situations that, in hindsight, may have been impacted by my gender, both good and bad.

I’ll start with where I am today, as CMO of Nearform. Being at Nearform has been an amazing experience as a woman in tech. I work on a majority female team, and sit on an executive team that has significant — at parity — female representation. We have an active Women's Guild that provides ongoing support, opportunities for learning, exclusive speakers and more. So I feel really fortunate to work in an environment where women are not only treated equally, but also given the space to celebrate our uniqueness.

I think a lot of diversity starts with seeing yourself in the roles you want to have. I've been very lucky to work at several companies where women comprise half or even more of the leadership team, and that's always been inspiring.

One thing I'd say is mentors are incredibly helpful for women in tech. As you know, many cities or sectors have close-knit ecosystems, and networking is critical to meet new people, as potential employers, clients, partners, or more. I personally have both male and female mentors, which I find helpful. It’s vital to have someone you can not only relate to, but who also brings a unique point of view and helps you mitigate your own blind spots.

Of course, there exists a reality where different individuals or groups may not have the same opportunities as others due to any number of reasons. I'd also say that, from a framing perspective, limits are temporary. Hearing “A woman's never done X,” or “There's no way we'll get to equal representation from Y per cent” is an antiquated and, frankly, damaging way of thinking. 

I’ve certainly faced my share of subtle or outright discriminatory comments and situations — and it’s up to us as women to find ways to not only speak truth to power in these individual instances, but also to internalise and learn from these experiences. As a result, we as women have strength beyond what we can at times imagine.

I truly believe there are no limits to what a woman can do, and often the first step to achieving a specific goal is that general ethos and mindset.