GIS files

Importing asset data

Projections

Here’s how projections work in the case of GIS files: 

• If the .prj file is not included in the Shapefile files, this layer is ignored.
• If the projection info is not declared in the GeoJSON file, it is assumed that the projection is canonical: WGS84 (EPSG:4326), as GeoJSON standard remarks.

GIS file names

To create a model, Qatium only requires you to import a pipe layer, but we recommend you import as many asset types as available to make the best of the platform.

In order for each asset to be visible on the map, it needs to be correctly assigned to a GIS file.  

During the import, Qatium will categorize your valid GIS files (GeoJSON and Shapefiles) automatically.

Here’s how this automatic categorization works: 

1. If a filename contains the aliases "no tratada",  "non potable", "air valve", "ventosa", the layer is categorized as ignored.

Example:

2. After this first step, the categorization takes into account the following conditions:

• If a file contains linear elements (lines or polylines) it is considered a pipe layer.
• If a file contains polygons or multipolygons with a DemandPattern property in some element (or with a zone alias in the filename), it is considered a zone layer.
• For files containing point or multipoint assets, we search for aliases in the file name. 

If there’s more than one asset that match, we associate the file to the asset type with the highest position in the table below:

Important:
Aliases are insensitive to case, accents and more than one space. For example, if you write zone instead of Zone the platform will still process it correctly.

Naming asset IDs 

You can provide IDs to your assets and Qatium will do its best to maintain them, as long as they meet certain conditions. 

Asset IDs must:

• include up to 31 valid characters 

All characters are considered valid, including space, tab and line break.

Qatium will automatically shorten a longer ID to 31 characters and replace any unidentified character with a number.  

• be unique

If 2 or more of your assets have the same ID, Qatium will assign that ID to the asset with the higher priority in the hierarchy list below: 

The rest of the assets will be renamed in the following way:

original ID + # (separator) + sequential number 

For example: 

Pipe1, Pipe1→ Pipe1, Pipe1#1 

Junction1, Junction1, Junction1 → Junction1, Junction1#1, Junction1#2

GIS fields

To help you create your model from GIS data successfully, we’ve created a table detailing the GIS fields considered by Qatium for each asset type.

Tip from Q:
Qatium validates your GIS data before creating a model from it, ensuring that the data uploaded is coherent. You can read more about Data Validation below.

Hydrants 

Just like an existing EPANET model, when creating a model from GIS data, you can specify which junction layers represent hydrants on your model. 

There are 2 ways to do it:

• By adding H_ as a prefix to the asset ID of the relevant junction layers
• By adding the word hydrant (or any of the accepted aliases) to the name of the relevant junction layers

Important:
“Hydrant” can be replaced by one of the following aliases: boca, faucet, fire, hidrante, hydrant, incendio, riego, salida, spigot

Laterals

You can also define lateral pipes in your network, so that you can visualize and filter them in the Network elements panel. 

To do so, specify “group” as field and “lateral” as attribute in your imported pipes.

For example, in your GIS attributes table, laterals would look like this:

Customer points 

Adding customer points to your network is very useful, as it allow you to:

• see the impact of an operation to the customers, when running scenarios 
• visualize customer consumption
• add consumption info to the simulation to improve the accuracy of the network
• get Water balance insights, especially related to efficiency 

You can do it in the following 2 ways:

• Using GeoJSON files: add “Customer”as an attribute to the “Group” field of your imported junction file.

For example:

• Using CSV files: you can export all the relevant data as a CSV file and then upload it directly to Qatium. There is no specific format required for your file but it should include longitude and latitude values, allowing Qatium to determine the coordinates.

          From here on, Qatium takes charge of transforming the data so that you can see them in your                     network. Here’s how this works:

1. Potential coordinate columns are identified. They should match a certain pattern, i.e. numbers between 180 -180 (for longitude) and 90 -90 (for latitude), with a decimal point (. or ,) and at least one number after. Next the candidate columns are checked and filtered and matched to longitude/latitude based on being within the bounding box of the network.
2. Asset IDs are established by searching for columns with unique values and selecting the first one.
3. The projection is assumed to be the default GIS projection (WGS84).
4. All the remaining data in the CSV are added to the assets, and they are processed the same way as properties coming in GIS files.

Thresholds

You can set thresholds for your assets when importing your GIS files. These values will then be compared with the live data. If the live data values exceed the threshold values for more than 1 hour, they are considered an issue, triggering an alert. If you don’t preset the threshold values, no alert will be raised.  

To add a threshold to an asset, add a new property called “WarningThresholdMin” (for minimum threshold) or “WarningThresholdMax” (for maximum threshold), as shown below:

Additional background layers

You can also add a layer containing any kind of geometry to your network. This can be useful to display buildings, labels etc.

In order to do so, you need to:

1. contact Q
2. upload your files (either Shapefiles or GeoJSON)

Once added, the new layer can be seen in the background of your network in a blue shade

For example, buildings would show like this:

Important:
If you experience any trouble while importing some background or zone layers involving demand zones, critical infrastructure, treatment plants, or any kind of boundary or label (like buildings, neighborhoods, city, region, restricted areas, etc), please get in touch with Q.

Recategorize layers

You can always recategorize your file layers any time after the import. 

Using the Network layer panel, you can select what type of asset represents each source or ignore it. 

Importing simple control rules

Qatium supports 3 kinds of simple rules as defined in EPANET during a GIS import (GeoJSON or Shapefile). 

You can include your rules in a field named [“rule”, “rules”] for pipes, valves and pumps.

Here’s what you need to know in order to import your simple control rules successfully:  

1. Qatium accepts a short version of control rules: the object type (here LINK) and ID followed by a list of its different settings separated by a separator:

Link 1 properties: LINK 1 CLOSED AT TIME 2, OPEN AT TIME 3

Will be interpreted as: LINK 1 CLOSED AT TIME 2 LINK 1 OPEN AT TIME 3

2. Qatium accepts control rules defined for other elements: any valid rule (containing the object type (here LINK elements) and ID, will be applied to the mentioned object regardless of the location of the rule:

Link 1 properties: LINK 2 CLOSED AT TIME 2, LINK 2 OPEN AT TIME 3

Will be interpreted as: LINK 2 CLOSED AT TIME 2 LINK 2 OPEN AT TIME 3

E.g.: A rule for Link 2 located in Link 1 properties, will be applied to Link 2 as long as the rule starts with “LINK 2”.

3. Qatium accepts multiple control rules separators: tab, comma, semicolon, break line. If the control rule is not valid, it will be included in the model and in the .inp file as a comment, in case the model is downloaded:

Link 1 properties: LINK 2 CLOSED AT TIME 2, LINK Qatium OPEN AT TIME 3

Will be interpreted as: LINK 2 CLOSED AT TIME 2 ; [not apply] LINK Qatium OPEN AT TIME 3

Important:
- Please import all GIS files at the same time. You can specify the Shapefile encoding in .cpg files, and for GeoJSON files, encoding shall be UTF-8 (as the JSON standard remarks).
- When uploading shapefiles, please upload .shp, .shx, .dbf and .prj files.

Data enhancement

If you don’t have all the optimal GIS data available, don’t worry. 

Qatium performs some data enhancement during your GIS data import. Q - our Smart Assistant - will filter, validate and infer from your existing data to build a working hydraulic model.

Data filtering

To accurately reflect your currently active network, Q filters any abandoned and proposed elements from your GIS dataset, except for zone layers.

Qatium will rely on the lifecycle property of all assets to discard them. Q will look for GIS fields named "life", "lifecycle", "d_lifecycl".

Here you have a list of the aliases for the lifecycle property:

Important:
Fields and attributes are not case sensitive.

Unit system detection

To determine the unit system of the network, Qatium performs checks on the diameter field of pipe files, which consist of 3 stages:  

1. Filtering: we accept values within the range from 1 to 4,000
2. Classification by file: we check if each file can belong to US Customary when

• the value of the maximum diameter is less than 150
• the value of the minimum diameter is less than 15

If neither of the conditions is met, the file is set as International System.

3. Setting the network unit system: if any of the pipe files was set as International System, the network unit is set as International System. Otherwise it is set as US Customary.

Geometry data filtering

Ignoring objects without geometry

Objects without coordinates in their geometry are discarded, which means that they won’t be represented in the model.

Discarding overlapping nodes  

When there are overlapping nodes (nodes with the same coordinates, with a 0.1 m tolerance), only the first element is preserved, according to the following order of priorities:

1. SUPPLY SOURCE
2. TANK
3. PUMP
4. VALVE
   1. PRV
   2. PSV
   3. FCV
   4. PBV
   5. GPV
   6. TCV
5. JUNCTION
   1. Hydrant family
   2. Junction family

Rounding coordinates

For the purpose of reducing file sizes, Qatium rounds the coordinates of all objects to 8 decimal places.  

Data validation 

Qatium validates your GIS data before creating a model from it, ensuring that the data uploaded is coherent. Qatium will read the layer attributes and make sure that each value complies with the validation rules listed in the table below. 

If a GIS attribute doesn't pass the validation stage, it will be ignored and we’ll replace the discarded values with default values.

Validation rules

Default attribute values

All attributes with unreadable or invalid values are set with the following default values in the defined unit system:

Important:
For pipes only: the mode is the most repeated value. Qatium will first try to calculate the mode of pipe diameter for each file. If successful, the value of the mode is used as the default value for pipes from a specific file.

Data inference

Qatium will try to interpret the GIS data imported and assign an asset type to each uploaded file. But if you can’t see all of your assets in your model, it is most likely that Q couldn’t recognize part of your dataset.

Here's how inference works for each asset:

 What's next?
➡️ Zones