Testing Geoportals: Services and Clients per ISTQB Process

Testing geoportals works best when you follow the ISTQB test process, from planning and control through to test closure, and treat services and clients as separate test objects. The quality characteristics that matter are functionality, usability, efficiency and security. On the service side, standards such as WMS come into play, along with dedicated checks for geodata and coordinate transformations.

Testing Geoportals With the ISTQB Test Process

Why Geoportals Need Their Own Testing

Many public authorities and companies that work with spatial information are building geoportals as a core part of their spatial data infrastructures (SDI). Well-known examples include Geoportal-Bund in Germany and geoland.at, the geoportal of the Austrian federal states. Some GI vendors offer SDI “frameworks”, “toolkits” or “suites”, but a software solution that fits an individual portal is not something you can buy off the shelf.

The main reason is that geoportals are not built on a greenfield site. They have to be integrated into existing environments, such as e-government infrastructures, each with its own set of constraints. On top of that, this kind of software is not in mass use; it tends to be developed, or further developed, project by project. So the operator of a new geoportal cannot simply trust that they will receive stable standard software. More likely, they will need to test the software together with the vendor and qualify it that way. The cost of this assessment through testing can easily reach around 30% of a software project’s total cost, and in some cases more. Against that background, this article explains which software tests are relevant for geoportals and how to carry them out.

What a Geoportal Is

In the narrow sense, a geoportal is a website that bundles access to distributed (base) services: map, data and catalog services. To do this, geoportals provide various clients (map viewers in particular), administration tools and, in turn, services of their own. These portal services make the base services connected to the portal available to external applications in processed form, which is how portals are linked with each other.

For users of these clients, the added value comes mainly from the portal aggregating the responses of the base services. Especially when the clients run in the browser, as is common today, the portal has to display maps from different Web Map Services together, for example, or perform coordinate transformations that the base services do not offer. The portal must also be able to combine the responses of base services into service chains (for example, showing a map for a geographic location).

The Test Project

Tests and reviews play an important role throughout the entire life cycle of a software application. During development, the application goes through unit, component and integration testing. Later, system and acceptance tests check whether the requirements and specifications have been implemented. Once the software is in use, regression tests and retests during maintenance (updates, patches) keep the quality in place. Behind all these kinds of testing sit a wide range of methods, processes and tools for checking different quality characteristics of the software, such as functionality, efficiency or usability.

Established norms and standards, rules and processes help run test projects in a structured, methodical way and assure the quality of the software. One possible foundation is the International Software Testing Qualifications Board (ISTQB), whose “Certified Tester” certification scheme gives testers and test managers tools, test processes and a shared terminology.

The approach presented here targets system and acceptance testing, because the business requirements take center stage at these levels. That said, the test levels before and after them are just as important for assuring the quality of a geoportal.

For a test project to run in a structured and efficient way, two things are needed: the test project has to be positioned within the overall software development and acceptance process, and a test process has to be established inside the test project. In this situation, a test process based on ISTQB is a good fit. The following sections describe each phase of the test project.

Test Planning and Control

As with any test project, testing a web-based geoportal starts by setting up the technical and organizational conditions for the test activities. A key part of this is writing a test plan. The ANSI/IEEE 829 standard (since superseded by ISO/IEC/IEEE 29119-3) provides a comprehensive structure for planning the test project and making it transparent. This gives everyone involved a common basis on which the whole test project is built. A test plan defines the following:

• Test objectives: Different test levels have different objectives. The objective of an integration test, for example, is to uncover interface defects. The objective of a system test is to validate the complete implementation against the requirements.
• Exit criteria: To know when a test phase has reached its objective, you need exit criteria. These can be various coverage measures, such as “all priority 1 test cases and at least 90% of priority 2 test cases executed”.
• Acceptance criteria: To assess the system at the end of the test project, you need criteria that describe the minimum conditions for putting the system into operation.
• Test strategy: For each test level or test type, define which methods and tools will be used to design and execute test cases.
• Framework conditions: Among other things, this defines which requirements form the basis of testing, in other words what the system is tested against. That can be existing specification documents such as business specifications, or standards such as ISO 19139 that the geoportal has to comply with.
• Identifying, structuring and prioritizing the test objects and functions: A web-based geoportal can be broken down into various test objects, such as map clients, interfaces, search functions and so on. These need to be identified and prioritized by criticality.
• Identifying test data
• Test levels and phases: This defines which types of testing are performed and which activities each test phase, such as test case specification, requires.
• Description of the test infrastructure (test environment, test workstations, hardware and software, tools)
• Defect management (defect classes, defect workflow)
• Suspension criteria and resumption requirements
• Identifying risks and hazards
• List of contacts, responsibilities and resources
• Identifying training needs for the test team
• Scheduling the test activities (availability of testers and services)

Alongside the test plan, test control is set up to deliver the current status of testing at all times. Even during test case specification, it can show how many test cases are still to be written. During test execution, it gives an overview of executed and open test cases, test case coverage and defects found. Defect management also needs to be established so that defects are documented traceably and none of them get “lost”.

The information from test control and defect management can be turned into interim and status reports. That keeps testing transparent at all times and lets problems surface early.

Test Analysis and Design

Next, the testers analyze the requirements documents and derive and specify test cases from them.

This is where the methods for test case design and test case reduction defined in the test plan come in. The test cases are documented in a test management tool. The documentation of test scenarios and test cases includes at least:

• A unique ID for the test case
• Test idea and test objective
• Test case description and expected result
• Test data for the test case
• Test case priority
• Reference to the requirement the test case was derived from
• Creation date and author of the test case

When analyzing the requirements, you need to distinguish between the different quality characteristics to be tested, each of which gets its own test cases. For a web-based geoportal, several characteristics typical of web applications are relevant, including:

• Functionality
• Usability
• Efficiency
• Security

Functionality

Functional testing mainly answers one question: “Does the geoportal as developed and delivered provide the functions defined in the requirements, completely and correctly?” For a geoportal in particular, the test objects include not only standard functions but also very complex ones, for example around coordinate transformations. Typical test objects in a geoportal include:

• File functions
• Administration functions
• Map clients
• Coordinate transformation
• Gazetteer
• Catalog service
• Interfaces

What to look out for in each of these test objects is covered in more detail in the second part of this article.

Usability

Usability testing assesses how easy the geoportal is to operate and use. It can cover the following topics, among others:

• User groups: Who will use this geoportal? How much help and explanation is needed, or how efficiently users can move through the portal, depends on who is supposed to use the system. The requirements will differ depending on whether subject matter experts or members of the public work with the portal. Checking whether these requirements have been met is the job of the usability test.
• Style guide compliance: This checks whether the geoportal was developed entirely according to the specified style guide.
• Browser compatibility: Which web browsers and operating systems does the geoportal have to work with? Given the complex structure of a geoportal, it may be enough to support only the core functions.
• Accessibility: Accessibility, meaning that the application can be used by everyone regardless of physical or technical capabilities, needs close scrutiny in a geoportal. Because it uses map material, full accessibility is not possible. So it has to be clear from the outset which requirements and functions must be accessible.

Efficiency

Efficiency testing examines how the geoportal behaves under load and overload, for example when many users access the portal at the same time. Web applications need particularly close attention here because of the sheer number of potential users. Again, the requirements need concrete targets, such as how many users to expect at the same time or over a given period and how quickly the geoportal must respond to requests under specific load conditions. How the system recovers after an overload is another possible aspect to test.

When running these tests, keep in mind that factors you cannot influence will affect the results, for example when WMS services from other providers are integrated.

For efficiency testing of web applications in particular, there is a wide range of commercial and open source tools that help both with generating load and with analyzing the data.

Security

Security is especially important for web applications and regularly makes headlines. The system, and above all the data, must be protected against unauthorized access and modification. Each user may see only the data they are authorized to see. Security is a broad topic that requires a high level of technical understanding. Problems in the infrastructure (such as the firewall, the network or the web or database server) or newly discovered security holes in applications can leave the geoportal open to attack. Security testing is usually done by specialists who understand the technical background. It is also very important to repeat security tests at regular intervals, because updates, configuration changes and newly discovered security holes can introduce new risks.

Besides these technical aspects, security also has to be looked at from the application’s point of view. The question here is: Does every user get only the content through the geoportal that they are entitled to use?

The prerequisite for this kind of testing is that users are grouped and assigned rights. The tests first check whether the user is authenticated correctly. After authentication, the geoportal maps users to their rights (authorization). For Web Map Services, rights can apply not only to layers but also to scale ranges, spatial extent or queries for attribute data (GetFeatureInfo request). For Web Feature Services, they apply to feature types and their attributes.

Whether these access rights are implemented correctly generally needs to be checked on two levels. The first is the description of the portal services. At interface level, the capabilities of the services must already be filtered according to user permissions; for example, the WMS capabilities contain only the layers the user is allowed to see. On the client side (to stay with the same example), only those layers are offered for selection. This level of security corresponds to the concept known as client-based security.

The second test level covers direct requests to the portal services that bypass the portal clients, in other words at interface level. The geoportal must block unauthorized requests to services and respond with an appropriate message.

In preparation for test execution, the need for test data is also identified while analyzing the requirements and creating the test cases. This test data must be available before test execution starts. It will be necessary to create test data manually or automatically, or to modify existing test data, with automatic generation being preferable. Another way to provide test data is to extract it from existing systems and anonymize it if necessary.

The test data is stored alongside the test cases in a traceable way, so that each test case can be repeated with the same data.

Test Implementation and Execution

Once the test infrastructure and the test cases are ready, the testers execute the test cases. During the test phase, functional tests as well as efficiency, usability and security tests are carried out. For a geoportal, make sure that all required external services are available during testing.

For each test case, the following information is logged during execution:

• Date and time of execution
• Tester: Who executed the test case?
• Execution status (passed, failed, possibly blocked)
• If it fails, the failure: How does the actual result differ from the expected result? Are wrong map contents or wrong information displayed, for example?

When a defect is found, it has to be logged in defect management. In addition to a precise description and metadata such as the version of the geoportal in which the defect occurred, the defect is assigned a defect class that describes its severity. The defect classes are documented in the test plan.

If defects found in the version under test have been fixed, a confirmation test (retest) and a regression test are carried out.

Evaluating Exit Criteria and Reporting

At the end of testing, the logged results are evaluated, checked against the acceptance and exit criteria and given a final assessment in a test summary report. The outcome of the report is a recommendation on whether the geoportal can go live or be accepted.

If no such recommendation can be given, the reasons for this must be listed in detail in the report.

Test Closure

An important activity at the end of the test project is analyzing and critically reviewing the entire test effort, so that the experience and insights feed into test process improvement and future test projects.

Finally, the components, drivers, test data and so on used for testing are archived so that the tests can be repeated or traced later if needed.

Testing Services and Clients

This part describes selected tests and test ideas for the functionality of geoportal clients as well as for services and interfaces.

Testing Functional Requirements

Because a geoportal is complex, the functionality of its services, interfaces and clients carries particular weight. That calls for a close look at the required functions and at the norms and standards behind them. The following sections go into this part of a test project in more detail.

Services and Interfaces

The architecture of a geoportal is based on loose coupling between the base services and the geoportal on the one hand, and between the portal services and external applications on the other. This requires each component to support defined interfaces. For geo services, these interfaces are described by the Open Geospatial Consortium (OGC) as Implementation Specifications and may be specified further through profiles. Compliance with these specifications is vital for portal components that handle external communication, so it plays a prominent role in testing.

The tool of choice for testing compliance with OGC specifications is the OGC’s CITE program (Conformance & Interoperability Test & Evaluation), which has since been completely revised. Among other things, it offers tests for various services such as Web Map Services (WMS), Web Feature Services (WFS) and Catalog Services (CSW). Information on the procedure and the terms of use is available on the CITE home page.

From a portal’s point of view, however, there are some particularities that limit the use of CITE tests. First, the CITE server needs full access to the services under test, which can be a problem in development environments or secured networks. Second, the CITE scenario requires test data provided by the OGC to be loaded into the services under test. Portal services, however, usually do not access data of their own but cascade to external services. So most CITE tests cannot be applied to portal services. The CITE program does not cover testing of client components either. Nor can CITE handle tests of profiles or tests of functional requirements tailored to the conditions of a specific geoportal. That means you have to define your own test ideas, based on CITE where possible.

Map Service (WMS)

The map service is the most frequently used service in a geoportal, which makes it the central object of testing.

The tests here are based on the three WMS interfaces GetCapabilities, GetMap and GetFeatureInfo. With few exceptions, all interface tests should call the interface directly, for example by entering a URL in the browser, to rule out defects in the WMS clients. The service’s response is then compared with the expected result. The exact structure of the requests is beyond the scope of this article.

For the GetCapabilities interface, test ideas like these are useful:

• Validity of the Capabilities.xml for the WMS version
• Correct behavior during version negotiation (responding with the requested version, otherwise with the highest implemented version number)
• Correctness of all OnlineResource URLs listed
• Completeness of the service metadata (This test case matters little for the actual functionality of the service. In interoperable environments, however, catalog services evaluate this metadata, for example, so it should be complete in the WMS capabilities.)

For map services, GetMap is the central interface. Testing it for correct behavior takes a lot of work, so only a few key test ideas are listed here:

• Robustness of the service when connected base services fail (This case should be triggered deliberately by shutting down one of the base services.)
• Availability of the layers listed in the capabilities
• Support for the image formats and coordinate reference systems (SRS) listed in the capabilities
• Correct interpretation of the parameters for transparency and background color
• Delivery of a map in the width/height ratio requested by the client (regardless of the width/height ratio of the bounding box)
• Testing the WMS coordinate transformation

As a rule, the capabilities specify scale ranges (ScaleHints) for some or all layers, for reasons of security, performance and/or marketing. In that case, tests should check that these are respected. This requires some manual calculation to create GetMap requests with the right content by hand. The bounding boxes calculated along the way can, however, be reused in other test cases.

Another group of tests concerns how the layers are rendered. Test ideas here include:

• Can each layer be rendered in every layer style listed in the capabilities?
• Are layers and styles matched correctly according to their order in the GetMap request?
• Can layers be requested in their default style through an empty STYLES parameter (STYLES=) or null values in the style list (e.g., STYLES=,,,)? Can these notations be combined with named styles?

If SLD-capable services are to be integrated into the geoportal, additional tests check whether the relevant SLD parameters (e.g., SLD=) are “passed through” to these services and whether the resulting map is rendered in the requested style.

For the GetFeatureInfo interface, first check whether attribute queries work for every layer marked with the attribute queryable=“1” in the service’s capabilities. As with GetMap, all listed output formats for presenting information should be available. If the corresponding parameter (INFO_FORMAT) is omitted, the output must still be in one of the formats listed in the capabilities.

In addition, test for every interface that it works without errors both with only the mandatory parameters and with optional parameters added, and that it returns a valid result even without vendor-specific parameters.

Data Service

In geoportals, data services answer queries for geographic objects such as places, addresses or regions (Gazetteer Service) and/or deliver raster or vector data (Web Coverage Service or Web Feature Service). There is no OGC standard for gazetteer services yet; the relevant paper currently has the status of a Best Practice document.

It treats a gazetteer service as an application profile of a WFS. If a geoportal’s gazetteer service is to use standardized interfaces, a WFS is therefore the only option. For using and processing the results, the key question besides interface conformance is whether the service delivers correct results. The following test ideas were chosen with that in mind; they refer only to version 1.0.0 of a Basic WFS, currently the most widely implemented version.

For the GetCapabilities interface of a WFS, questions like these arise:

• Is the capabilities response valid, and does it match the version number given?
• Are feature types returned correctly, and does each have a valid SRS assigned?
• Is at least GML2 listed as a ResultFormat?

For the DescribeFeatureType interface, first check whether the server response delivers a valid GML schema and whether all feature types of the WFS are described even when the TYPENAME parameter is omitted.

In terms of content, testing the DescribeFeatureType interface needs special attention to one point: only those feature attributes should be described that are meant to be “made public” through the service.

For the GetFeature interface, you then test whether data is really returned only for these attributes. To do this, set the PROPERTYNAME parameter in the GetFeature request to * and, in a second test, leave it out entirely. At a minimum, the following test ideas should also be used:

• Validity of the returned GML document against the DescribeFeatureType XML schema, including the imported GML schemas
• Support for the filter encoding methods listed in the capabilities
• Correctness of query results when using filter encoding (e.g., by comparing with SQL queries)

The last test takes considerable effort, because filter encoding queries can be very complex and meaningful queries have to be built from the available data, with their results then checked. Be aware that these tests in particular can only ever be spot checks.

In addition, check for every WFS operation that both the HTTP request methods GET and POST are supported. GET requests can be entered as a URL in the browser, but POST requests cannot. A purpose-built web page or application with an input form that is sent to the server via POST helps here (for a web page, this requires write access on a web server).

Coordinate Transformation Service

Geoportals need coordinate transformations in several places. This section focuses on server-side transformations; client-side transformations are covered elsewhere.

This functionality is usually not exposed as an OGC web service but used only internally. The tests therefore target the results of the transformation, not interfaces.

Coordinate transformations matter most when map images are transformed for WMS requests. Portal components have to perform transformations whenever clients request an SRS that the base services do not support.

The transformation routine of the portal WMS can be tested with a separate reference data set. A good test datum is a rectangle roughly the size of the geographic area for which the transformation is to be tested.

This rectangle is stored in different layers in each SRS of interest, with its target coordinates calculated by hand in each case. A GetMap request sent from the browser requests maps in the different SRS, with the layer that does not need transforming serving as the reference. A faster alternative is to use the bounding box as the reference.

If the geoportal offers geo search functions (places, addresses and the like) and visualizes the search results, a coordinate transformation is needed whenever the SRS of the map differs from that of the search results. If the search runs through a WFS, the coordinate transformation is done either by a dedicated coordinate transformation service (mandatory up to WFS version 1.0.0) or by the WFS itself (from version 1.1.0, although a transformation service may still be called internally). Here, too, the coordinate transformation service is used only inside the geoportal to convert individual (GML) features into the requested SRS and is not offered externally. In this case, you do not need to test the service’s interfaces, only the correctness of the transformation result. That is naturally the case when the map and the visualized GML features overlay correctly.

When interpreting the test results, keep in mind that coordinate transformations have different levels of geometric accuracy. To identify transformation errors, you need to know the potential accuracy of the transformation method and parameters in question. When converting between Gauss-Krüger zones (e.g., EPSG codes 31466 to 31469), the ellipsoid does not change and exact formulas are applied, so accuracy is high. For transformations that require a datum shift, however, such as from Gauss-Krüger coordinates on the Bessel ellipsoid to UTM coordinates on the GRS80 ellipsoid (EPSG codes for Germany: 25832 and 25833), the conversion is not exact but based on transformation parameters. For Germany, there are several parameter sets that differ in accuracy for a given (geographic) area of use. As a result, coordinates can differ by a few meters if target and actual coordinates were calculated with different parameter sets.

Administration Functions

The administration options of a geoportal include configuring access rights to the various data, services and functions, as well as integrating and arranging a wide variety of services, layers and specialist data in the portal clients.

Clients

For geo services, standardization largely fixes the functionality. For clients, it can vary widely from one geoportal to the next because of different requirements and implementations. For that reason, only selected test objects and test ideas can be covered here.

Map Clients

Depending on how they are built and used, map clients can offer a varying range of functions. This can also depend on the technology: a Java- or AJAX-based client will offer more options than an HTML client that may not even use JavaScript.

The following typical map client functions can serve as test ideas:

• Map navigation: zooming in and out of map sections by specific factors, and panning.
• Calculations: area and length calculations, as well as displaying coordinates for specific points.
• Map composition: adding, removing and reordering layers or services.
• File functions: saving settings such as layer combinations, map sections and sizes on the portal or locally, so they can be loaded again later to continue working. The same applies to saving map sections and specialist information.
• Print functions: the print functions of web browsers are not up to many situations, so map client print functions often offer different print sizes, resolutions and information, or deliver the output directly as PDF files.

Client components also need coordinate transformations:

• Transforming image coordinates into world coordinates with every map navigation, to create the bounding box of the map request
• Transforming the bounding box when the user changes the SRS
• Displaying world coordinates

In the simplest case, the transformation of the bounding box coordinates is checked using the map client’s coordinate display. If that is not implemented, network protocol analysis tools can be used to inspect the incoming and outgoing HTTP traffic at the client, and with it the transformed coordinates.

The client’s coordinate display can be checked with a simple comparison of target and actual values at control points.

Clients for Address and Place Search

These clients forward queries about geographic regions, addresses or places to distributed gazetteer services, and the results are usually visualized directly in the geoportal’s map client. The clients are therefore expected to formulate the requests correctly. In practice, this is checked indirectly by comparing the results of queries made through the client with those of HTTP requests sent to the service.

You should also test whether different ways of entering search terms lead to correct answers. For example:

• Exact search using quotation marks (e.g., “Zittauer Gebirge”)
• Handling of wildcards (? or *)
• (Optional) case sensitivity
• Searching with umlauts or their substitutions (e.g., ä -> ae, ß -> ss)
• Combining search terms with AND, OR, NOT
• Missing address suffixes such as Strasse, Weg, Ufer
• Fuzzy search for misspelled terms, among others

Further test scenarios concern how the results are presented:

• Correct rendering of the search results (e.g., location shown correctly on the map)
• Compliance with the usability criteria defined for the portal (e.g., a sorted list of hits the user can choose from when there are several, “meaningful” messages for invalid input or an empty result set, a cap on the number of hits or, alternatively, a prompt to refine the search criteria)

Clients for Searching Data and Services

Catalog clients for finding data and services usually query several metadata catalogs based on ISO 19115/19119. This requires the client to support searching via the application profiles implemented in the connected catalog services, such as the German profile of the ISO 19115/ISO 19119 application profile for OGC Web Catalogue Services (CSW 2.0).

As with address and place search, the list of hits for a query has to be checked for correctness. You can do this by sending identical queries through the geoportal’s metadata search client and directly to the connected external metadata catalogs, for example through their clients or (better) as SOAP requests, and comparing the two responses. Here, too, different input variants must lead to correct results.

Another group of tests concerns how the search results are presented in the client. Test ideas should check the results for:

• Correct content of the displayed results (compared with direct queries to the distributed metadata catalogs)
• Completeness of the displayed metadata elements (depending on the agreed element set of the result: BRIEF, SUMMARY or FULL)
• Usability (e.g., sensible sorting of the list of hits, links to service providers, or the option to add services found to the geoportal)

Conclusion

Geoportals are spreading quickly and are used by people with no GIS background, they serve critical areas such as internal security and disaster management, and they rest on a service-oriented architecture whose particularities have to be taken into account. All of this creates specific quality requirements for the software behind them. This two-part article covered the software tests needed to meet those requirements. In particular, it showed how general IT testing know-how and GI application know-how work together when specifying and executing the tests, and how that contributes to the success of the project.

Looking ahead, test automation in particular will open up further ways to make the necessary tests even more efficient.

Frequently Asked Questions

Why Can’t Geoportals Be Procured as Off-the-Shelf Standard Software?

Geoportals are rarely built from scratch; instead, they are integrated into existing environments, such as e-government infrastructures, with their own specific requirements. Furthermore, such software is not widely used off-the-shelf but is customized for specific projects. Even the GDI frameworks, toolkits, and suites offered in 2007 did nothing to change this. The operator must therefore test the solution together with the provider to validate it.

What percentage of a software project’s costs is attributed to testing?

The 2007 article estimated that testing costs accounted for about 30 percent of the total costs of a software project, and in some cases even more. This proportion is the reason why testing should be planned as a separate project (with a test plan, test management, and defect management) rather than being carried out as an afterthought at the end of development.

At which test level are the business requirements for a geoportal verified?

At system and acceptance testing levels, as these levels focus on validation of the overall implementation against the requirements. The upstream and downstream phases remain important: unit, component, and integration testing identifies interface errors, among other things, during development, while regression and retests ensure quality during updates and patches in live operation.

Can a geoportal be fully accessible?

No. Because it uses map data, full accessibility is not achievable. Therefore, it must be clarified before testing which requirements and functions must be implemented in an accessible manner. The usability test also checks whether the level of assistance and navigation are appropriate for the intended user groups, whether the specified style guide has been followed, and which browsers and operating systems are supported.

What factors distort load testing for a portal that integrates third-party services?

Factors outside one’s own control affect the measured values, such as integrated WMS services from other providers. Efficiency tests only become meaningful when based on requirements: how many users are expected simultaneously or over a given time period, and how quickly the portal must respond under specific load conditions. Recovery after overload is also part of the testing process.

How do you verify that users only see the geodata authorized for them?

On two levels. First, the capabilities of the portal services must already be filtered according to user rights, so that a WMS only reports the permitted layers and the client only displays these for selection. This corresponds to client-based security. Second, requests that bypass the portal clients must be tested: The geoportal must block unauthorized requests and respond with a message.

Are the OGC’s compliance testing procedures sufficient for certifying a geoportal’s services?

No. In 2007, the OGC’s CITE program offered tests for Web Map Services, Web Feature Services, and catalog services, but these required the test server to have full access to the services and the integration of provided test data. However, portal services typically access external services in a cascading manner. Client components, profiles, and project-specific functional requirements were left out anyway; these require separate test ideas.

How do you distinguish between genuine errors and normal inaccuracies in coordinate transformations?

You must know the potential accuracy of the respective method and its parameters. Between Gauss-Krüger strips (EPSG codes 31466 through 31469), the ellipsoid does not change; strict formulas are applied, and accuracy is high. For datum transitions, such as from Bessel to GRS80 (EPSG 25832 and 25833), calculations are performed using transformation parameters. For Germany, there are several sets of parameters, and deviations of a few meters are possible.