ASCII standard

The listed formats and following data layouts are accepted by the ASCII interpreter in SHOP. Comments can be inserted if the first character on the line is #.

To identify the data layout of the following lines, the ASCII interpreter in SHOP reads up to five identifiers on the line preceding the actual data. They must be ordered as in the table below.

Name	Comment
Object type	Refers to which object type the data describes
Attribute	Refers to which attribute the data describes
Object name	Name identifying the object the data should be applied to
Second object_name	Name identifying the second object, if data is given as a relation between two objects
Third object_name	Name identifying the third object, if data is given as a relation between three objects

Reading and writing with pyshop

ASCII data can be read into SHOP with the read_ascii_file function in pyshop.

from pyshop import ShopSession

shop = ShopSession()
shop.read_ascii_file(file_path='basic_model.ascii')

There is not an equivalent dump_ascii function in pyshop, but a SHOP case can be written to ASCII files with the command print ascii.

Defining the time horizon

The duration of the optimization period is defined by the start_time and the end_time. The start_time is the first point in time that is included in the optimization horizon, and the end_time is the first point in time that is after the optimization horizon. In the example below the optimization period will be one week. Note that OPTIMIZATION is a valid alias for GLOBAL_SETTINGS in the ASCII format.

#Object_type  Attribute
 GLOBAL_SETTINGS time
#Start_time End_time
 2021010100 2021010800

SHOP accepts both fixed and time-dependent resolution in the optimization period. The time_resolution uses the standard TimeSeries format, but must be given directly after the start and end times, and before any other time series data is input to SHOP. In the example below the time resolution is 1 hour for the first day, and 3 hours for the remaining days.

#Object_type  Attribute
 GLOBAL_SETTINGS time_resolution
#Id Number Start_time Time_unit Period Data_type Y_unit Pts
 0  0      2021010100 HOUR      8760   -1        HOUR   2
#time       y
 2021010100 1
 2021010200 3

Object definitions

A new object should be declared with the declaration statement:

 RESERVOIR declaration Reservoir1

If the object has not been declared when the first object attribute is being set, a warning will be written to the log file.

The deprecated attributes keyword is also used to create some object types and specifies some attributes at the same time:

RESERVOIR attributes

The deprecated reservoir object creation also sets max_vol, lrl, and hrl:

#Object_type Attribute  Object_name
 RESERVOIR	 attributes	Reservoir1
#Id Water_course Type Maxvol Lrl    Hrl
 0  0            0    300.00 400.00 450.00

PLANT attributes

The deprecated plant object creation also sets outlet_line, main_loss, and penstock_loss:

#Object_type Attribute  Object_name
 PLANT       attributes Plant1
#id water_course type bid_area prod_area num_units num_pump
 1  1            0    1        1         2         0
#num_main_segm num_penstock time_delay prod_factor outlet_line
 1             2            0          0.000       100.000
#main_loss
 0.00001
#penstock_loss
 0.00001 0.000011

GENERATOR attributes

The deprecated generator creation structure has two formats. The first format must be used if the type of generator is given as 0 or 1 (regular generator), while the second format must be used if the type of generator is given as pelton generator. The generator is identified by the name of the plant and the following number of the unit. In the example below attributes are set for generator 1 and generator 2 in Plant1. The first generator is not a pelton unit, while the second one is.

Non-pelton format

#Object_type Attribute  Object_name Second_object_name
 GENERATOR   attributes Plant1      1
#id type penstock nomprod minprod maxprod start_cost
 0  0    1        45.000  15.000  50.000  2100.0

Pelton format

#Object_type Attribute  Object_name Second_object_name
 GENERATOR   attributes Plant1      2
#id type   penstock start_cost no_needle_comb
 0  pelton 2        2100.0     3

NEEDLE_COMB attributes

For pelton generators, each needle_combination object also has a deprecated creation structure. In the example below, attributes are set for needle combination 1 of generator 1 in Plant1. The needle combination number is given as the third object name, while the generator number is given as the second object name.

#Object_type Attribute  Object_name Second_object_name Third_object_name
 NEEDLE_COMB attributes Plant1      1                  1
#id type nom_prod min_prod max_prod
 0  0    120      70       120

PUMP attributes

The deprecated creation structure for the pump object is shown below. The pump is identified by the name of the plant and the following number of the unit. In the example below attributes are set for pump 1 in Plant1.

#Object_type Attribute  Object_name Second_object_name
 PUMP        attributes Plant1      1
#id type penstock nomprod start_cost minprod maxprod
 0  0    1        45.000  5000.0     80      100

GATE attributes

The deprecated creation structure for the gate object is shown below:

#Object_type Attribute  Object_name
 GATE        attribute  Gate1
#Id water_course type time_delay num_parallel_gates gate_slack
 1  1            0    0          1                  0

TUNNEL attributes

The deprecated creation structure for the tunnel object is shown below

#Object_type Attribute  Object_name
 TUNNEL      attributes Tunnel1
#loss_factor start_height end_height diameter length
 0.00016     90           90         3        2022

JUNCTION attributes

The deprecated creation structure for the junction object is shown below. The tunnel losses are given first for tunnel1 and on the next line for tunnel2.

#Object_type Attribute  Object_name
 JUNCTION    attributes Junction1
#id type num_inputs altitude junc_slack
 0  0    2          80.0     0
#Tunnel loss
 0.0004
 0.0002

JUNCTION_GATE attributes

The deprecated creation structure for the junction_gate object is shown below. The tunnel losses are given first for tunnel1 and on the next line for tunnel2.

#Object_type Attribute  Object_name
 JUNCTION    attributes JunctionGate1
#id type num_inputs altitude junc_slack
 0  0    2          80.0     0
#Tunnel loss
 0.0004
 0.0002

CREEK_INTAKE attributes

The deprecated creation structure for the creek_intake object is shown below:

#Object_type  Attribute  Object_name
 CREEK_INTAKE attributes Creek1
#Id main_tunnel_loss tunnel_loss creek_level cap_mode
 0  0.0005           0.0001      456.2       0

CONTRACT definition

The deprecated creation structure for the contract object is shown below. It is similar to the deprecated MARKET format, but using the definition keyword and without the connection to a production area.

#Object_type Attribute  Object_name
CONTRACT	 definition	Contract1
#Number of XY functions for this contract
 3
#Start time for this contract
 2021010100
#Id Number Reference Npts x_unit y_unit
 0  1      0.0       4    MW     NOK/MWH
#x    y
 -500 100
 -200 140
 100  150
 500  180
#Start time for this contract
 2021010101
#Id Number Reference Npts x_unit y_unit
 0  1      0.0       4    MW     NOK/MWH
#x    y
 100  160
 500  190
 700  205
 1100 210
#Start time for this contract
 2021010102
#Id Number Reference Npts x_unit y_unit
 0  1      0.0       4    MW     NOK/MWH
#x   y
 100 130
 300 145
 400 190
 575 220

(ascii:setting-attributes)

Setting attributes

Single values

A single value can be input to SHOP by preceding it with a line identifying the object type, attribute name and object name that it refers to. This works for integer, double, and string attributes. Below is an example of setting integer, double, and string attributes for three different objects:

## Integer value
# Object_type Attribute  Object_name
 PLANT        min_uptime  Plant1
# value 
 120 

## Double value
# Object_type Attribute  Object_name
 BATTERY      max_energy  Battery1
# value 
 10.2      

## String value
# Object_type Attribute  Object_name
 MARKET       market_type  Market1
# value 
 ENERGY 

int_array

Attributes with the datatype int_array are defined in ASCII as a downward list with an initial value for the size of the array:

# Object_type Attribute  Object_name
 PLANT        gen_priority  Plant1
# length
 4
# values 
 3
 1
 2
 4 

double_array and string_array

Attributes with the datatype double_array and string_array are defined as a list of values on the same line with spaces between the values. Note that the initial length specifier needed in the int_array is not present:

# Object_type Attribute              Object_name
 PUMP         discrete_droop_values  Pump1
# values
 2.2 4.1 5.6 12.2

XY-curve and XY-curve array

The structure is used to store a single XY and XY-array function that consists of the elements given in the table below.

Name	Comment
Id	Id number (not used internally, set to 0)
Number	Unit number (not used internally, set to 0)
Reference	Reference value (depends – if not used, set to 0)
Pts	The number of data pairs
X_unit	MW, METER, PERCENT etc.
Y_unit	MW, MM3, PERCENT etc.
pairs of (x,y)	Related values of data pairs x and y

The Id and Number fields are not used by SHOP. We simply set these values to 0. The Reference is used for XY-array where multiple XYs are specified for different reference values. It is, for example, used for efficiency curves to hold the reference head (meters) for the given curves. However, reference is not always used for each individual case. If a XY function does not need reference, we can simply set the value to 0. The X_unit and Y_unit must match the SHOP-definition for the given function. Below is an example of an XY function about the relationship between water volume and level of a reservoir.

#Object_type Attribute  Object_name
 RESERVOIR   vol_head   Reservoir1
#Id Number Reference Pts X_unit Y_unit
 0  0      0         5   MM3    METER
#x      y
 0.00   860.00
 5.07   870.00
 10.34  878.00
 21.10  890.00
 30.36  898.00

All attributes in the attribute table with datatype XY can generally be input using the ASCII XY-format described here, and datatype XY-array by repeating the ASCII XY-format including reference. Exceptions from this rule are listed in the [exception table].

TimeSeries (old name: TXY)

This data structure is used for the time-dependent values. The structure consists of the elements listed in the table below.

Name	Comment
Id	Id number (not used internally, set to 0)
Number	Unit number (not used internally, set to 0)
Start_time	Start time of first data in the series
Time_unit	SECOND, MINUTE, or HOUR
Period	Period length, if data is repeatable
Data_type	-1 or 0 (tells how to interpret series values)
Y_unit	MW, MM3, PERCENT etc.
Pts	The number of data pairs
pairs of (time,y)	Related values of data pairs time and y

The Id and Number fields are not used by SHOP. The Start_time is the time of the first y-value. The time format can be 17 digits yyyymmddhhmmssmmm <year, month (1-12), day (1-31), hour (0-23), minute (0-59), second (0-59) millisecond (0-999)>. If fewer digits are used, it is assumed that the omitted digits are all zero. The Time_unit is MINUTE or HOUR according to the definition of given data. If Period = 0, complete data must be given for the whole planning period; If Period > 0, the last value will be repeatedly used until the end of planning period; If data is specified in hours for a given day and the Period = 24, the data will be used repeatedly for each day in the optimization period. The Data_type describes how the data for a point of time between the specified two points will be interpreted. If Data_type = 0, the current value is a linear interpolation based on the two points on either side or an extrapolation before or after the end; If Data_type = –1, the current value is set to the value of the nearest previous value (most commonly used). The Y_unit must match the SHOP-definition for the given function. The time values are start times for the corresponding y values. The first t-value must always be equal to the Start_time. Most time series in SHOP can be deactivated by writing NaN as the value. The table below shows an example of a TimeSeries function that is a minimum production constraint on the discharge of a hydropower plant. The constraint is active until January 1st 18:00.

#Object_type Attribute    Object_name
 PLANT       min_p_constr Plant1
#Id Number Start_time Time_unit Period Data_type Y_unit Pts
 0  0      2021010100 HOUR      24     -1        MW     4
#time       y
 2021010100 200.00
 2021010108 400.00
 2021010112 300.00
 2021010118 NaN

All attributes in the attribute table for the TimeSeries datatype can generally be input using the ASCII TimeSeries-format described here. Exceptions from this rule are listed in the [exception table].

XYT

The XYT data structure is used for XY data with a time reference. See XY for syntax example.

SY

The datatype is used for string-float key-value pairs.

#Object_type Attribute    Object_name
 BUSBAR       ptdf Busbar1
#string       y
 AC_line1 0.4
 AC_line2 0.6
 AC_line3 0.4

Connecting objects

The CONNECT statement in the ASCII format is used to couple objects together. Both directional connections (reservoir -> plant) or and bi-directional logical connections (plant <-> discharge_group) use the same format. The general syntax is defined on the following line:

CONNECT <from_object_type>/<to_object_type> <from_object_name> <to_object_name>

In the example below, the plant “Plant1” is connected as output from reservoir “Reservoir1”.

#Object_type Attribute       Object_name Second_object_name
CONNECT      RESERVOIR/PLANT Reservoir1  Plant1

Connecting spill and bypass gates

The deprecated gate object uses the special “object_types” BYPASS and SPILL instead of GATE in the connect statement when a bypass or spill gate is connected to a reservoir:

# Object_type Attribute        Object_name   Second_object_name
CONNECT      RESERVOIR/BYPASS Upstream_rsv  Bypass_gate1
CONNECT      RESERVOIR/SPILL  Upstream_rsv  Spill_gate1

However, the gates should use the GATE object type when connecting to any downstream object:

# Object_type Attribute      Object_name   Second_object_name
CONNECT      GATE/RESERVOIR  Bypass_gate1  Downstream_rsv  
CONNECT      GATE/RESERVOIR  Spill_gate1   Downstream_rsv  

Regular gates that are not bypass or spill gates should use the GATE keyword for both upstream and downstream connections.

Deprecated ASCII structures

The following deprecated ASCII structures will be removed in a future SHOP version, see the deprecated functionality overview for more details.

MARKET

The market is described as an XY-table that contains options for buying and selling electricity. The x-value is the trading volume in MW and y-value is the price per MWH. SHOP does not need to know which monetary unit is used, it is the users responsibility to use the monetary unit consistently for all values input to SHOP. SHOP designates monetary units by KRONER or NOK, but this does not mean that Norwegian Kroner has to be used as the actual unit.

Each of these market functions has a Start_time that defines from which interval the function is used. One function is valid until the next one become active.
The market options must be applied in order from selling to buying sorted by the size of the options. The amount for selling is negative and the one for buying is positive.

The market structure can be divided into an area description. One market for each area must be supplied. A market description for one area is shown in the table below. Take for example the description for the first hour. This description applies an option for selling at price 170.070 KRONER/MWH with a 500 MW capacity limit. Option Two tells that it is possible to buy maximum 500 MW capacity for the price of 170.072 KRONER/MWH.

#Object_type Area_no
 MARKET      1
#Number of XY-table for Area 1
 5
#Start_time for 1st XY-table
 2021010100
#Id Number Reference Pts _unit Y_unit
 0  0      0         2    MW   KRONER
#x       y
 -500.00 170.070
 500.00  170.072
#Start_time for 2nd XY-table
 2021010107
#Id Number Reference Pts _unit Y_unit
 0  0      0         2    MW   KRONER
#x       y
 -500.00 200.420
 500.00  200.422
#Start_time for 3rd XY-table
 2021010112
#Id Number Reference Pts _unit Y_unit
 0  0      0         2    MW   KRONER
#x       y
 -500.00 180.290
 500.00  180.292
#Start_time for 4th XY-table
 2021010116
#Id Number Reference Pts _unit Y_unit
 0  0      0         2    MW   KRONER
#x       y
 -500.00 171.380
 500.00  171.382
#Start_time for 5th XY-table
 2021010120
#Id Number Reference Pts _unit Y_unit
 0  0      0         2    MW   KRONER
#x       y
 -500.00 165.110
 500.00  165.112

PLANT_OUTLET

Using an ASCII input file the plant_outlet structure is shown below. In contrast to other objects, the name of the plant_outlet is given at the second position of the line, normally used for attributes.

#Object_type Name
PLANT_OUTLET Outlet1
#num_main_seg
 2
#Main_segment_loss no_of_plants plant_names
 0.001             2            Plant1 Plant2
 0.002             3            Plant1 Plant2 Plant3

STARTRES

Using an ASCII input file the startres structure is shown below. In contrast to other objects, the position normally used for the attribute name is here used for the number of upcoming lines with start reservoir data. The unit can either be METER or MM3 and must be the same for all reservoirs in one STARTRES definition.

#Object_type num_of_rsv unit
 STARTRES    2          METER
#Object_name value
 Reservoir1  872.62
 Reservoir2  694.20

INITIAL_STATE

Using an ASCII input file the initial state structure is shown below. After INITIAL_STATE follows the number of aggregates /pumps with an initial state. The next line contains the plant name, generator or pump object type, the generator- or pump number and finally the initial state.

#Object_type Number_of_aggregates_and_pumps
INITIAL_STATE 3
#Plant_name GENERATOR_or_PUMP unit_number initial_state
 Plant1     GENERATOR         1           0
 Plant1     GENERATOR         2           1
 Plant2     PUMP              1           1

MULTI_OBJECT_DATA

Using an ASCII input file the multi object data structure is shown below. MULTI_OBJECT_DATA marks the start tag of a multi object constraint, and /MULTI_OBJECT_DATA defines the closing of the multi object data. A multi object constraint contains sections defined by specified start and end tags. The multi object must contain an object list section and a data value section, and may contain one interval and penalty cost section.

#Object_type      Attribute Object_name Second_object_name
MULTI_OBJECT_DATA keyword   sense       name
OBJECT_LIST
object_type1 name1
objevt_type2 name2
...
/OBJECT_LIST
TIME_INTERVAL
start_time end_time
/TIME_INTERVAL
PENALTY_COST unit
UP value unit
DOWN value unit
/PENALTY_COST
DATA_VALUE
data value unit
/DATA_VALUE
/MULTI_OBJECT_DATA

Valid values for MULTI_OBJECT_DATA:

• Type of constraint :

  • Sum for each time step: sum_discharge, sum_production_time_step, spinning_reserve, total_reserve, contract_volume_time_step;

  • Sum/Average in a time interval: average_discharge, sum_production_time_period;

  • Ramping specific: power_ramping, discharge_ramping, storage_ramping;

• Sense of constraint :

  • Less than: L or l (if the constraint is of type a+b≤c),

  • Greater than: G or g (constraints of type a+b≥c),

  • Equal to: E or e (constraints of type a+b=c); Name of restriction : string

  • The name should be unique within the set of names used for constraints.

The sense of a constraint has a different meaning for ramping. Namely, L stands for ramping up and G stands for ramping down. E is not used in combination with ramping.

The object list is a list of the objects involved in the constraint. The list can be of any length, excet for ramping which only accepts single objects).

Valid values for OBJECT_LIST:

• object_type: PLANT, GATE, RESERVOIR

• name: The name of the plant, gate and reservoir, respectively.

  OBJECT_LIST MyList object_type1 name1 objevt_type2 name2 … /OBJECT_LIST

Valid values for TIME_INTERVAL:

• start_time: yyyymmddhhmmssmmm

• end_time: yyyymmddhhmmssmmm

Default value (if time_interval is not included) is the whole optimization period.
Valid values for PENALTY_COST:

• value: double

• unit: NOK_H_M3_S — (money_units/hour)/(m3/s); NOK_MWH — money_units/ mega_watt_hour. Valid values for DATA_VALUE:

• data_value: double

• unit: M3SEC; MW; MWH; MM3; METER.

The allowed combinations that can be used in multi object format are listed below.

Attribute	Object_types	Unit	Penalty_unit
sum_discharge	PLANT,GATE	M3SEC	NOK_H_M3_S
sum_production_time_step	PLANT,CONTRACT	MW	NOK_MWH
spinning_reserve	PLANT,CONTRACT	MW	NOK_MWH
total_reserve	PLANT,CONTRACT	MW	NOK_MWH
contract_volume_time_step	PLANT,CONTRACT	MW	NOK_MWH
average_discharge	PLANT,GATE	M3SEC,MM3	NOK_H_M3_S
sum_production_time_period	PLANT,CONTRACT	MWH,MW	NOK_MWH
power_ramping	PLANT,CONTRACT	MW	NOK_MWH
discharge_ramping	PLANT,GATE	M3SEC	NOK_H_M3_S
storage_ramping	RESERVOIR	MM3,METER	NOK_MM3