Key Points

References

Reference_description_with_linked_URLs_____________________________________	Notes_________________________________________________________________


https://searchsoftwarequality.techtarget.com/tip/A-Selenium-IDE-vs-WebDriver-comparison Selenium IDE vs. WebDriver comparison pdf	Selenium IDE vs. WebDriver comparison pdf **

https://www.guru99.com/top-6-api-testing-tool.html	API Testing Tools list
https://www.postman.com/pricing/	Postman - Enterprise API Test tool with many features, easy to learn, use free for limited usage, 3 developers
https://curl.se/docs/ https://curl.se/docs/manual.html	Curl CLI for HTTP requests
https://github.com/httpie/httpie	Httpie - Modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more.
https://www.selenium.dev/downloads/ https://www.selenium.dev/documentation/ https://www.selenium.dev/documentation/overview/ https://www.selenium.dev/selenium-ide/docs/en/introduction/getting-started https://www.selenium.dev/selenium-ide/docs/en/introduction/command-line-runner	Selenium Web Driver browser client records Web sessions for test scripts, playback supports key OS, Languages add Appium for Mobile tests
https://spockframework.org/ https://www.baeldung.com/groovy-spock	Spock Spock is a testing and specification framework for Java and Groovy applications
https://www.gebish.org/ https://www.softwaretestinghelp.com/geb-tutorial-browser-automation-testing-using-geb-tool/ https://medium.com/automationschool/geb-spock-the-best-of-breed-solution-for-all-your-ui-automation-needs-94f01c998f18 Concept - Geb and Spock Tests	Geb - a web testing automation framework. It is build as a wrapper over Selenium WebDriver which makes it ideal for automated browser testing Geb Tutorial – Browser Automation Testing Using Geb Tool is a good tutorial **
https://cucumber.io/ https://cucumber.io/docs/guides/10-minute-tutorial/	Cucumber
stackshare ompare of cucumber and spock http://ytheohar.blogspot.com/2015/07/bdd-with-cucumber-vs-spock.html https://www.donaldmcintosh.net/blog/goodbye-cucumber-hello-spock https://www.slideshare.net/wakaleo/cucumber-and-spock-primer cucumber-and-spock-primer-slides-170424130138.pdf	Compare Cucumber and Spock BDD Tests frameworks This BDD comparison shows Spock syntax is much simpler to read, code cucumber and spock primer is on slideshare
https://cucumber.io/docs/installation/	cucumber installation
https://medium.com/@mlvandijk/getting-started-with-cucumber-in-java-a-10-minute-tutorial-586652d2c82	10 minute cucumber tutorial
https://medium.com/@prathore28647/cucumber-vs-bdd-behavior-driven-development-a5dec8d64bcd	cucumber vs BDD concepts article
https://medium.com/@mlvandijk/getting-started-with-cucumber-in-java-a-10-minute-tutorial-586652d2c82	getting started with cucumber in Java
cucumber in groovy
https://www.slideshare.net/rapaul/cucumber-on-the-jvm-with-groovy groovy-cucumber-slides-onthejvmwithgroovy-rapaul-110324134849-phpapp02.pdf link	Cucumber concepts on groovy slides - 2011
https://javadoc.io/doc/io.cucumber/cucumber-groovy/latest/io/cucumber/groovy/package-summary.html	groovy cucumber javadoc
https://blog.jetbrains.com/idea/2012/11/cucumber-for-java-and-groovy-in-intellij-idea-12/	cucumber for java and groovy in intellij tutorial 2012
https://github.com/cucumber/cucumber-jvm-groovy	cucumber jvm groovy - 2020 updated
Behavior Driven Development with Cucumber, Groovy and Grails video
https://cucumber.io/docs/installation/groovy/	install cucumber for groovy

https://www.eclemma.org/index.html https://www.eclemma.org/userdoc/coverageview.html https://www.eclemma.org/resources.html	Jacoco - Java code coverage tool

Key Concepts

Potential Value Opportunities

Potential Challenges

Candidate Solutions

Javascript Test Tools List

Mocha

Jasmine

Karma

Instanbul

compare the above for Angular / React testing, functional, code coverage etc

Java Test Tools List

API Test Tools List

An API or Application programming interface is a collection of software functions and procedures through which other software applications can be accessed or executed. In API Testing you use software to send calls to the API, get output and log the system's response. For Agile development, Api Testing becomes important as shorter development cycles put more pressure on automated testing.

Here is a list of top Web Services Testing Tools

1) SOAtest

Parasoft SOAtest is the industry-leading API testing solution, which simplifies the process of creating automated end-to-end test scenarios across multiple layers of modern applications (i.e. mobile, REST APIs, SOAP services, Microservices, databases, Web UIs, ESBs, or mainframes) from a single intuitive interface.

Rapid and scriptless test creation: Automatically generate tests from a service definition to test design of an API. Use drag and drop tools for data driving, looping and performing complex assertions
Save time building tests: SOAtest plug-in leverages artificial intelligence to automatically convert UI tests into scriptless API test scenarios
Change management: SOAtest can identify service schema changes and automatically update your library of tests
Maximum scalability: Integration for existing SDLC test frameworks and CI infrastructures. Seamless integration with load testing and service virtualization
Robust and flexible integration: Broad support for 120+ protocols and message formats including JMS, MQ, TCP, File, Copybook, FIX, EDI and many more

2) API Fortress

API Fortress is the most powerful API testing and monitoring solution for REST and SOAP APIs.

Comprehensive test generation with a single click.
Web-based collaborative tool for teams. Works within your browser and requires no downloads.
Intuitive UI that is easy to use at any skill level.
Simple one-click automation. Test during development and deployments (CI/CD), and monitor functional uptime.
Robust notification and data integrations.
Works in the cloud, on-premises, or both in an ad hoc manner.

Hundreds of enterprises have already made the switch. More power, less confusion. The easiest way to automate your API testing and monitoring strategy.

3) Checkly

Checkly is a fresh new API monitoring and site transaction monitoring platform. With its great UI and dashboards, monitoring your crucial API endpoints and vital site click paths has never been easier.

Flexible and deep API monitoring with support for GraphQL, XML, JSON and extensive request customisation.
Puppeteer scripted browser clicks flows with assertions and screenshots.
Email, SMS, Slack and Pagerduty alerting integrations.
Great public, customisable dashboards & status pages for customers or in the office.
Full scripting flexibility: use Node.js code to set up custom requests or reset test data.
Easy CI/CD integration for ad hoc tests and validation.

4) APImetrics

APImetrics active API monitoring gives you up-to-date, accurate reporting of all the APIs and micro-services you supply or depend on, and how they impact your business.

Test API collections in the real world, from real application servers, hosted on the cloud servers your clients use including AWS, Azure, Google and IBM
Identify content errors, passing ‘failures’ and networking configuration problems
State of the art AI trained on a database of over 1 billion API calls to identify hard to spot issues and take you straight to problems
Integrated analysis and reporting including real time alerts, dashboards and quality scoring with Cloud API Service Consistency (CASC) Scores, a credit score for your API, benchmark your API to your industry

5) Ping API

Ping-API is API testing allows to write test script in JavaScript and CoffeeScript to test your APIs. It allows inspecting HTTP API call with a complete request and response data. For any failures, the user gets a notification through email, slack or Hipchat.

Features:

Ping- API to schedule test in every minutes or hour
Support for writing script to set request headers, body and URL parameters. It supports for writing script to validate response headers and body
Validate CRUD flow and log in to Ping API

Download link: https://ping-api.com/

6) HP QTP(UFT)

It provides an extensible framework helpful in executing and building the functionality of headless system that do not have a user interface. It helps to test the headless technologies like Databases and Webservices, JMS, etc. By using the API test conversion tool, you can convert soapUI tests to UFT (QTP) API tests.

Download QTP

7) vREST

vREST provides an online solution for automated testing, mocking, automated recording and specification of REST/HTTP APIs/RESTful APIs.

It provides an exhaustive tool to quickly validate your REST APIs
It delivers zero defect web applications with less effort in API testing
To validate your web application no skilled resources are required, and it can generate documentation for your API specifications
API mocks can be created in vREST with the help of Mock Server Functionality. User can directly start developing frontend using mock HTTP requests

Link to Vrest

8) Postman

Postman is a plugin in Google Chrome, and it can be used for testing API services. It is a powerful HTTP client to test web services. For manual or exploratory testing, Postman is a good choice for testing API.

With Postman, almost all modern web API data can be extracted
You can write Boolean tests within Postman Interface
You can create a collection of REST calls and save each call as part of a collection for execution in future
Unlike CURL, it is not a command line based tool, which makes this tool hassle free of pasting text into command line window
For transmitting and receiving REST information, Postman is more reliable

Link to Postman

9) HttpMaster

It is a web development tool to automate web application testing, including API testing, service testing and website testing. It is primarily used as web API test tool to automate testing of web API calls.

HttpMaster is a best choice for API testing as it has got

Several http methods (GET, POST, DELETE etc.)
Dynamic parameters of various data types to make batches of different API requests
Various validation types and advanced validation expressions

Download HttpMaster

10) Rest-assured

It’s a popular framework to test REST services in Java.

Learn more about Rest-Assured

Download HttpMaster

10b> Cucumber BDD DSL and Selenium Web Driver

https://arsfutura.com/magazine/cucumber-bdd-with-selenium-and-java/

arsfutura.com-Cucumber BDD with Selenium and Java.pdf link

arsfutura.com-Cucumber BDD with Selenium and Java.pdf file

driver.findElement(By.name("q")).sendKeys("Youtube");

WebElement searchIcon = driver.findElement(By.name("btnK"));
searchIcon.click();

Cucumber DSL

Scenario Login
 Given I'm on login page
 And I enter my email
 And I enter my password
 When I click login button
 Then I should be logged in

Guide to Cucumber Selenium

https://www.browserstack.com/guide/automation-using-cucumber-selenium

Automation Testing accelerates the software development lifecycle. Furthermore, on adopting the Behavior Driven Development (BDD) approach to development, different stakeholders like developers, QAs, and non-tech teams can collaborate actively in the project.

The scenarios are written based on the expected behavior of the software and it is tested to check if it matches said scenarios.

11) Karate DSL - DSL for automated BDD tests without language coding needed

Karate is a new API testing framework base on cucumber library. Karate DSL allows testers to write meaningful tests for web service using a domain-specific language.

Features:

It supports configuration switching/staging, multi-threaded parallel execution
Allows testing and generating reports just like any standard Java Project
Possible to write tests even for the non-programmers
The karate API testing tool allows re-use of payload-data and user-defined functions across tests.

Download link: https://github.com/intuit/karate

https://github.com/karatelabs/karate

https://www.softwaretestinghelp.com/api-testing-with-karate-framework/

karate beginner tutorial video

karate setup and install video

Can create automated, BDD tests with Karate using a DSL ( vs a language )

Karate framework follows the Cucumber style of writing the program which follows the BDD approach. The syntax is easy to understand by non-programmers. And this framework is the only API testing tool that has combined API Automation and performance testing into a single standalone tool.

It provides the users with the ability to execute the test cases in Parallel and perform the JSON & XML checks.

With this information, certain key points can be deduced to further understand the Karate tool in detail:

Karate is a BDD testing framework instead of a TDD.
It is designed to be easy for non-programmers. This feature is a game-changer as it allows for more use and access by many people regardless of their technical background or capacity.
It makes use of the Cucumber feature file and the Gherkins language to write the test which is very easy to understand.

TestNG - NextGeneration - more flexible than JUnit

https://www.guru99.com/all-about-testng-and-selenium.html

testng-2022-guru99.com-TestNG Tutorial What is Annotations amp Framework in Selenium.pdf file

https://testng.org/doc/documentation-main.html

TestNG is an automation testing framework in which NG stands for “Next Generation”. TestNG is inspired by JUnit which uses the annotations (@). TestNG overcomes the disadvantages of JUnit and is designed to make end-to-end testing easy.

Using TestNG, you can generate a proper report, and you can easily come to know how many test cases are passed, failed, and skipped. You can execute the failed test cases separately.

12) Rest Console

HTTP client and Request visualizer and constructor API testing tool. It helps developers to build, debug and test RESTful APIS.

Features:

Construct POST or PUT body via raw input
Easy query parameters creation
Add custom headers through intuitive UI
Keyboard navigation and shortcuts

Download link: https://github.com/ahmadnassri/restconsole

13) Hippie-Swagger

Hippie-swagger is a tool for testing APIS. It supports a clear assertion syntax, for extending test behavior. It also allows clean printing reports.

Features:

It will fail test whenever swagger documentation is either erroneous or missing
Readable and accurate assertation messages
Validated parameters, request, response, pats, etc.

Download link: https://github.com/CacheControl/hippie-swagger

14) Pyresttest

PyRestTest is a Python-based REST API testing platform. It supports tests in JSON or YAML config files. Therefore, there is no code needed.

Features:

It’s minimal dependencies, which allows easy deployment on-server for smoke tests/health checks
This API testing tool will return exit codes on failure
Pyresttest allows to generate/extract and validate mechanisms to build test scenarios

Download link: https://github.com/svanoort/pyresttest

15) Airborne

Airborne is an API automation testing tool used for testing Rest APIs.

Features:

Airborne is a programming framework, so it has no user interface apart from the text file to create code
To use airborne, you just need to remember a few key methods in the toolset and some ruby and rspec fundamentals

Download link: https://github.com/brooklynDev/airborne

16) JMeter

JMeter is used for functional API testing which includes needed to test an API. It also has features which help to boost the performance of API testing.

Features:

It can record web sessions for playback ( similar to Selenium WebDriver )
It can be used for both static as well as dynamic resources performance testing
It supports replaying of test results
It can automatically work with CSV files. This helps test team to produce unique parameter values for API tests

JMeter is a 100% Java application and should run correctly on any system that has a compliant Java implementation.

Download link: http://jmeter.apache.org/

https://github.com/apache/jmeter

Ability to load and performance test many applications/server/protocol types:

Web - HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET,...)
SOAP / REST Webservices
FTP
Database via JDBC
LDAP
Message-oriented Middleware (MOM) via JMS
Mail - SMTP(S), POP3(S) and IMAP(S)
Native commands or shell scripts
TCP
Java Objects

https://jmeter.apache.org/usermanual/get-started.html

in testing a Web application, then see the section Building a Web Test Plan. The other specific Test Plan sections are:

Once you are comfortable with building and running JMeter Test Plans, you can look into the various configuration elements (timers, listeners, assertions, and others) which give you more control over your Test Plans.

https://www.kiwiqa.com/selenium-vs-jmeter-an-in-depth-guide-for-testers/

Some of the potential cons of the JMeter are:

JMeter does not offer support to the AJAX and JavaScript requests.
Automation and Customization are difficult in the case of JMeter.

17) APIpray Inspector

Apiary allows monitoring the API during the design phase by capturing both request and response. It allows the user to write API blueprints and lets the user view them Apiary editor or Apiary.jo.

Features:

Role-based access control over API documents.
It allows to add and remove team members from API design projects
API Blueprint management dashboard

Download link: https://apiary.io/

18) SOAP Sonar

SOAPSonar is an API testing and diagnostics platform for SOAP, XML, REST-based web services. Use external sources like Excel, MS SQL, Oracle or any ODBC Database for automated tests.

Features:

Identifies Web Services vulnerabilities like Malware Threat and SQL Injection
Web Service Functional Testing with Success Rule Framework and Concurrent Client Load Testing
Native HP QC Integration, and supports integration with Hudson, Ant, and JUnit

Download link: http://www.crosschecknet.com/

19) API Science

API science allows monitoring the health, availability, and performance of web APIs. The tool enables monitoring Private, Partner and Public APIs. This tool allows the user to know if any API ever goes down so that needed action should be taken to bring it back up.

Features:

It is multi-step & Powered by JavaScript
Powerful reporting mechanism helps to get insights into historical trends and spot future issues.
Supports JSON, REST, XML, and Oauth
Helps to manage API supply chain

Download link: https://www.apiscience.com/

20) Apigee

Apigee is a cross-cloud API testing tool. It allows the user to measure and test API performance, supports and build API using other editors like Swagger. It offers security and governance policies across all APIs.

Features:

Allows to design monitor, deploy, and scale APIs
Easily create API proxies from the Open API Specification and deploy them in the cloud
Identify performance issues by tracking API traffic, error rates, and response times

Download link: http://apigee.com/

21) Tricentis

Tricentis is a robust web services testing tool. Prominent API Testing features of Tricentis Tosca are –

It supports a wide array of protocols including HTTP(s) JMS, AMQP, Rabbit MQ, TIBCO EMS, SOAP, REST, IBM MQ,NET TCP
It integrates into the Agile and DevOps Cycle
It uses model-based test automation that makes script maintenance easy.
Enables end-to-end testing as API tests can be used across mobile, cross-browser, packaged apps, etc.

Tricentis’ 400+ customers include global names from the Top 500 brands such as ExxonMobil, HBO, Whole Foods, Toyota, Allianz, BMW, Starbucks, Deutsche Bank, Lexmark, Orange, A&E, Vantiv, Vodafone, Telstra and UBS.

Download link: https://www.tricentis.com/automated-software-testing-tool-trial/

22) SOAP UI

The most widely popular tool for API testing in the world, SoapUI allows you to test REST and SOAP APIs with ease – as it has been built specifically for API testing.

Quick and Easy Test Creation: Point-and-click, drag-and-drop, functionality makes complicated tasks (like working with JSON and XML) simple
Powerful data-driven testing: Load data from Excel, files, and databases to simulate the way consumers interact with your APIs
Reusability of Scripts: Reuse your functional test cases as load tests and security scans in just a few clicks
Seamless Integrations: Integrates with 13 API management platforms, supports REST, SOAP, JMS, and IoT

SoapUI Pro is used by thousands of leading companies around the world, including: Apple, Microsoft, Cisco, Oracle, HP, NASA, eBay, MasterCard, Intel, FedEx, and Pfizer

Download link: https://smartbear.com/product/ready-api/soapui/overview/

Full Stack Open-Source Observability and Monitoring Solutions

For open-source alternatives to AppDynamics, consider SigNoz, a full-stack observability platform, or Prometheus, a widely-used monitoring system, often paired with Grafana for visualization. [1, 2, 3, 4, 5, 6, 7, 8, 9]

SigNoz: Is a full-stack open-source APM and observability tool

Here's a more detailed breakdown: [6]

SigNoz: [6]
- Is a full-stack open-source APM and observability tool. [6]
- Provides a unified UI for metrics, traces, and logs with advanced tagging and filtering capabilities. [1, 2]
- Can be used for effective API monitoring. [9]
- Offers distributed tracing as a major feature. [10]
- According to SigNoz, it is a perfect open-source alternative to AppDynamics. [1]
Prometheus: [8]
- Is a widely-used open-source monitoring and alerting toolkit. [4, 8]
- Integrates well with OpenTelemetry. [8]
- Is often used with Grafana for visualization. [4]
- Is known for its ability to collect time-series data and metrics. [4]
- According to Last9, it is a widely used open-source monitoring solution. [4]

Generative AI is experimental.

[1] https://signoz.io/blog/appdynamics-alternative/

[2] https://signoz.io/comparisons/appdynamics-competitors/

[3] https://medium.com/@MetricFire/prometheus-vs-appdynamics-cf833fb673f0

[4] https://last9.io/blog/top-5-dynatrace-alternatives/

[5] https://last9.io/blog/8-datadog-alternatives-for-2024/

[6] https://signoz.io/blog/apm-tools/

[7] https://www.cloudzero.com/blog/datadog-alternatives/

[8] https://www.kloudmate.com/alternatives/opentelemetry-observability-platforms

[9] https://signoz.io/blog/api-monitoring-tools/

[10] https://dev.to/signoz/top-11-mysql-monitoring-tools-in-2022-open-source-included-247m?comments_sort=latest

Prometheus: a widely-used open-source monitoring and alerting toolkit. Integrates with Open-Telemetry and Grafana

AppDynamics - a commercial tool for application performance management - consider open-source options instead

AppDynamics, now a part of the Splunk Observability portfolio, is a comprehensive solution for application performance management (APM) and IT operations analytics (ITOA).It empowers businesses to monitor, troubleshoot, and enhance the performance of their applications and infrastructure. [1, 2, 3]

Key Features: [2, 4]

Real-time performance monitoring and proactive issue resolution.
End-to-end visibility across the entire application stack for root cause analysis.
Correlation of application performance with business outcomes.
Support for cloud-native application observability in cloud environments.
Application security features for vulnerability detection and mitigation.
IT infrastructure monitoring, including servers, databases, and networks.
AI-powered insights through machine learning for automated tasks.
Integration with the Splunk platform for enhanced observability capabilities. [1, 2, 3, 4, 5, 6, 7, 8, 9]

Benefits: [2, 4]

Improved application performance and enhanced user experience.
Faster problem resolution through end-to-end visibility and AI-powered features.
Data-driven decision-making and improved business outcomes.
Streamlined IT operations with automation and centralized performance views.
Reduced costs associated with downtime and poor user experience. [2, 3, 4, 6, 7]

Generative AI is experimental.

[1] https://www.splunk.com/en_us/appdynamics-joins-splunk.html

[2] https://www.cisco.com/c/en_au/solutions/data-center/appdynamics-application-performance-monitoring.html

[3] https://en.wikipedia.org/wiki/AppDynamics

[4] https://accounts.appdynamics.com/trial

[5] https://www.wwt.com/article/what-is-appdynamics-ciscos-apm-solution-explained

[6] https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/digital-estate/app-dynamics

[7] https://docs.appdynamics.com/appd/24.x/24.1/en/appdynamics-essentials/appdynamics-concepts

[8] https://www.splunk.com/en_us/products/splunk-appdynamics.html

[9] https://docs.appdynamics.com/display/ONPREM/AppDynamics+Concepts

Browser not supported