SHOP Server REST API endpoints

This document describes the REST API endpoints that are exposed when shop_server_host has been set up as a service on a remote machine. HTTP requests can be made to these endpoint to run SHOP models and retrieve the results using the SHOP JSON data format. For testing purposes on local machines, starting the shop_server_host application in one terminal and sending requests with a small python script using the requests module from another terminal should work:

import requests

# Check that SHOP Server is running by using the GET method on the general path '/'
response = requests.get("http://127.0.0.1:8000/", json={})

# Should get "OK" back, see general endpoint section below
print(response.text)

Initial configuration

It is possible to specify a number of command line arguments when starting the SHOP Server application:

Option	Value type	Default	Description
-l, --listen	string	0.0.0.0	Bind network address
-p, --port	int	8000	Listening port
-m, --max-sessions	int	CPU_threads * 4	Max concurrent sessions
-t, --logging-endpoint-timeout	int	3	Timeout in seconds for logging endpoint
--shop-log-file	string	(empty)	Output file path for SHOP logs
--solver-log-file	string	(empty)	Output file path for CPLEX logs
--minimal-infeasible-problem-file	string	(empty)	Output file path for minimal infeasible problem data

Example usage:

./shop_server --listen 127.0.0.1 --port 9000

Note that setting the --max-sessions flag can be important when running many SHOP optimizations in parallel. Remember to carefully select how many threads CPLEX can use in each parallel optimization with the max_num_threads SHOP attribute so that it matches with the available hardware resources. The total number of parallel runs multiplied with max_num_threads should not be larger than the total number of threads on the system, otherwise CPLEX from one session will fight to use the same threads as CPLEX in another. This will be detrimental to calculation times.

Specifying the SHOP and CPLEX log file paths is possible, but the SHOP log messages can also be sent to a specified logging endpoint. When models are infeasible, CPLEX will dump a minimal infeasible problem to the path pointed to by minimal_infeasible_problem_file. This attribute can also be specified in the --minimal-infeasible-problem-file option to SHOP Server.

General

SHOP Server implements the following general endpoints for testing purposes:

Method	Path	Description	Request Body	Headers	Response (200)
GET	/	Health check to confirm SHOP server is running	(none)	(none)	OK\r\n\r\n (type is text/plain)

Session

SHOP Server has built in support for running several parallel SHOP optimization sessions. A session is required to hold the SHOP model itself.

Method	Path	Description	Request Body	Headers	Response (200)	Notes
GET	/session	Get information about the current session	(none)	session_id	SessionSchema	Returns 4xx/5xx if session not found
POST	/session	Create a new session	SessionSchema (without session_id)	(none)	SessionSchema (with session_id)	429 if too many sessions
POST	/session/cancel	Cancel the current session by aborting CPLEX and any further SHOP commands	(none)	session_id	(empty body)	4xx/5xx if session not found
DELETE	/session	Delete a session	(none)	session_id	(empty body)	422 if session_id missing/invalid
GET	/sessions	List all existing sessions	(none)	(none)	Array of SessionSchema

SessionSchema:

Field	Type	Description
session_id	int	Assigned by the server when a session is created
session_name	string	User-defined session name
log_file	string	Unused parameter

Example:

{
  "session_id": 1,
  "session_name": "day ahead bidding 1",
  "log_file": ""
}

Model

After a session has been created, a SHOP model can be created and solved in that session.

Method	Path	Description	Request Body	Headers	Query Params	Response (200)	Notes
PUT	/model	Upload a full model to the session and apply it	ModelSchema	session_id	(none)	Resulting ModelSchema filtered by only including input objects and attributes	The ModelSchema does not need to contain all data, multiple requests can be made to build up the model
GET	/model	Fetch model data, optionally filtered	(none)	session_id	objectType, objectName, attributeName, datatype, isInput, isOutput, includeTime, includeConnections, compressTxy	ModelSchema filtered by query params	All query params are optional filters, the first four are strings and the remaining are booleans
GET	/model/:object_type/information	Get information about a specific object type	(none)	session_id	verbose (bool, default false)	ObjectTypeInfoSchema	The verbose = true filter will include all attribute information, similar to the /model/information path
GET	/model/information	Get information about all possible attributes and object types	(none)	session_id	(none)	ModelInfoSchema	The same information that is provided in the attribute table on the SHOP Portal

The ModelSchema mentioned in the model endpoints should be on the SHOP JSON data format. The format has four root nodes (“time”, “model”, “connections”, and “commands”), but not all nodes must be present for a valid POST request if the user wants to build the model in stages. Setting the static topology data with connections first, then setting time resolution and time series data, and finally running commands is therefore a valid option.

ObjectTypeInfoSchema:

Field	Type	Description
object_type	string	The object type in question
instances	Array of strings	The object names of all instances of the object_type in the current model
attributes	AttributesSchema	A map where all existing attribute names for object_type are the keys and AttributeInfoSchema are the values

If verbose = false, the resulting AttributeInfoSchema is a single string which is the datatype of the attribute. When verbose = true, the AttributeInfoSchema becomes:

Field	Type	Description
attribute_name	string	The attribute name
legacy_datatype	string	The datatype of the attribute
is_input	bool	True if the attribute can be given as input
is_output	bool	True if the attribute is an output attribute
x_unit	string	The unit of the x coordinate
y_unit	string	The unit of the y coordinate
license_name	string	The name of the license required to get or set the attribute
full_name	string	Unused
data_func_name	string	Attribute name alias
description	string	A description of the attribute
documentation_url	string	Reserved for future use
example_url_prefix	string	Reserved for future use
example	string	Reserved for future use
default_value	string	String representation of the default attribute value
required	bool	Is true for attributes marked as required SHOP input

Example of ObjectTypeInfoSchema with verbose = false:

{
    "object_type": "reservoir",
    "instances": [
        "Rsv_1",
        "Rsv_2"
    ],
    "attributes": {
        "max_vol": "double",
        "lrl": "double",
        "hrl": "double",
        /* Remaining reservoir attributes... */
    }
}

Example of ObjectTypeInfoSchema with verbose = true:

{
    "object_type": "reservoir",
    "instances": [
        "Rsv_1",
        "Rsv_2"
    ],
    "attributes": {
        "max_vol": {
            "attribute_name": "max_vol",
            "legacy_datatype": "double",
            "is_input": true,
            "is_output": false,
            "x_unit": "MM3",
            "y_unit": "MM3",
            "license_name": "SHOP_OPEN",
            "full_name": "Max volume",
            "data_func_name": "",
            "description": "Maximum volume of the reservoir",
            "documentation_url": "https://docs.shop.sintef.energy/attribute-table.html",
            "example_url_prefix": "https://docs.shop.sintef.energy",
            "example": "",
            "default_value": "0.0",
            "required": false
        },
        /* Remaining reservoir attributes... */
    }
}

The ModelInfoSchema is a map where the object type names of all possible objects in SHOP are keys, and attribute maps are the values. Each attribute map has all possible attribute names for the object type as keys, and verbose AttributesSchemas as values. Unlike the ObjectTypeInfoSchema, instances of the object types are not reported.

Logging

It is possible to specify a logging endpoint in SHOP Server to catch the SHOP log messages.

Method	Path	Description	Request Body	Headers	Response (200)	Notes
POST	/logging/endpoint	Register a logging endpoint for this session	LogEndpointSchema	session_id	(empty body)	Subsequent logs will be sent as LogMessageSchema to the registered endpoint

LogEndpointSchema:

Field	Type	Description
endpoint	string	URL to which the session will send log messages

Example:

{
  "endpoint": "https://my_logging_endpoint.shop.com"
}

LogMessageSchema:

Field	Type	Description
message	string	Log message text from SHOP
level	string	Log level, either "INFO","WARNING", or "ERROR"
callerid	string	The session_name of the session sending the log message

Example:

{
  "message": "Head optimization on.\n",
  "level": "INFO",
  "callerid": "my_session_1"
}

Time resolution

The time resolution can be retrieved with a request to the dedicated “/time_resolution” endpoint.

Method	Path	Description	Request Body	Headers	Response (200)
GET	/time_resolution	Retrieve the time resolution data from the current session	(none)	session_id	TimeResolutionSchema

The TimeResolutionSchema is the same as described in the SHOP JSON data standard documentation under “Time”.

Utility commands

Some additional commands have been implemented under the /internal endpoint:

Method	Path	Description	Request Body	Headers	Response (200)	Notes
POST	/internal/GetLicenseInfo	Get information about the license file	(none)	session_id	Array of strings	The strings in the response array are of the form “SHOP_TUNNEL:Unlimited access”
POST	/internal/DumpYamlString	Dump the model to YAML format	DumpYamlSchema	session_id	string	The entire model is returned as a single string on YAML format

DumpYamlSchema:

Field	Type	Description
args	array of three bool parameters	The three parameters describe: if only input data should be included, if the TXYs should be compressed, if the connections should be compressed

Example:

{
  "args": [
        true,      
        true,
        false
  ]
}