Abstract

In this post, we explain how the TCAP-MAP protocol has been defined and extended in its successive versions, which led to some backward incompatibilities in the message decoding process. This is where pycrate comes to the rescue, providing an extended TCAP-MAP ASN.1 module that supports all MAP versions; it enables the encoding and decoding of any TCAP-MAP messages in a convenient and straightforward way.

Introduction to TCAP-MAP

TCAP-MAP is one of the main protocols used in 2G and 3G mobile core networks, and probably still the most used at operators’ interconnect to support roaming. MAP (Mobile Application Part) is a set of operations and data structures to support mobile subscribers mobility and services over 2G and 3G access networks and for both CS (Circuit Switch) and PS (Packet Switch) domains.

The TCAP-MAP protocol is mostly used (but not only) between MSC-VLR (Mobile Switching Center – Visited Location Register) and HLR (Home Location Register) in the CS domain, and SGSN (Serving GPRS Support Node) and HLR in the PS domain.

MAP is currently maintained by the 3GPP and continues to be extended regularly. It can be downloaded on the 3GPP website under the reference TS 29.002. It relies on TCAP (Transaction Capabilities Application Part) which defines generic call flows and generic message structures for any kind of application procedures. ITU-T is responsible for the TCAP specification under the following references: Q.771, Q.772, Q.773 and Q.774.

The Q.773 is of particular interest as it defines the TCAP message formatting and encoding; it has been stable for more than 20 years. In short, TCAP uses ASN.1 BER (Basic Encoding Rules) and specifies several types of generic message structures, whereas MAP defines specific operations and argument structures that get embedded into those generic TCAP messages.

TCAP-MAP messages are formally defined by using ASN.1. The ASN.1 definition for TCAP message structures can be found in the ITU-T Q.773 document (and also here). The ASN.1 definitions for MAP operations and argument structures are provided in section 17 of the 3GPP document. Those MAP ASN.1 definitions can be used to parameterize the TCAP ASN.1 definition, in order to produce a complete TCAP-MAP schema that will enable to encode and decode all possible messages for all the defined MAP operations.

This is exactly what is done within the pycrate_TCAP_MAP module within Pycrate, a Python library maintained by P1 Security that integrates an ASN.1 compiler and run-time. In order to get accustomed with this specific module, one can read the pycrate wiki dedicated to TCAP-MAP and TCAP-CAMEL (Customised Applications for Mobile network Enhanced Logic).

Basic structure of a TCAP-MAP message

A TCAP message can be of 5 different types: unidirectional (which is a standalone message) or begin, continue, end or abort (which are parts of a transaction). MAP does not make use of the unidirectional message and only uses messages for proper transactions, with at least a begin and an end message. Each message is made of 3 parts:

• TCAP transaction identifier(s),
• the TCAP DialoguePortion to indicate and eventually negotiate global parameters related to the whole TCAP transaction,
• the TCAP ComponentPortion to exchange various MAP operations information between the TCAP transaction initiator and responder.

DialoguePortion and ComponentPortion are optional, and may be missing in certain TCAP-MAP messages. The transaction initiator always sets an originating transaction identifier in the initial TCAP begin message. The transaction responder may set an additional transaction identifier, in case the transaction does not stop with the first response and further message exchanges are expected.

The DialoguePortion is often used to indicate which version of the MAP application-context is to be used between the TCAP transaction endpoints. Within a TCAP transaction, multiple MAP operations can be handled; under certain conditions, or for some MAP operations, this DialoguePortion can also contain subscribers’ identifiers involved in all MAP operations of the transaction.

The ComponentPortion contains a list of components, which are requests, responses or errors related to MAP operations. Around 80 different MAP operations are defined in the most recent MAP technical specification, such as UpdateLocation under operation code 2, CancelLocation under code 3, USSD under code 60…

For each operation, specific structures are defined for the request (ArgumentType), the response (ResultType) and potential errors (ParameterType). MAP operations are grouped into operation-packages, which themselves are grouped into application-contexts. A MAP application-context corresponds to a type of interface between two different core network equipment, or to a set of related operations. Here is an example of such a TCAP-MAP message exchange:

-- request from the VLR to the HLR
begin : {
  otid 'AA900487'H,
  dialoguePortion {
    direct-reference {0 0 17 773 1 1 1} -- dialogue-as-id --,
    encoding single-ASN1-type : DialoguePDU: dialogueRequest : {
      protocol-version '1'B -- version1 --,
      application-context-name {0 4 0 0 1 0 1 3} -- networkLocUpContext-v3 --
    }
  },
  components {
    basicROS : invoke : {
      invokeId present : 0,
      opcode local : 2,
      argument MAP-MS-DataTypes.UpdateLocationArg: {
        imsi '12257909013376F7'H,
        msc-Number '91606431129025'H,
        vlr-Number '91606431129025'H,
        vlr-Capability {
          supportedCamelPhases '1'B -- phase1 --,
          supportedLCS-CapabilitySets '111'B -- lcsCapabilitySet1 | lcsCapabilitySet2 | lcsCapabilitySet3 --
        },
        add-info {
          imeisv '9508534135112675'H
        }
      }
    }
  }
}

-- response from the HLR to the VLR
end : {
  dtid 'AA900487'H,
  components {
    basicROS : returnResult : {
      invokeId present : 0,
      result {
        opcode local : 2,
        result MAP-MS-DataTypes.UpdateLocationRes: {
          hlr-Number '912488234935F9'H
        }
      }
    }
  }
}

Meet the different MAP versions

The MAP protocol has been historically defined within the ETSI in the early 1990s, together with the first version of the technical specifications for GSM systems. Those early MAP specifications can be found under the ETSI GTS 09.02 reference, where one can find a scanned document from February 1992 providing the specification of the initial MAP protocol corresponding to the GSM phase 1 in version 3.0.0 (alias MAP version 1). This initial GSM infrastructure had support for only basic CS services (i.e. calls).

In this directory, one can also find the specification for the MAP protocol corresponding to the GSM phase 2+ in version 5.3.0 (alias MAP version 2). The GSM phases 2 and 2+ correspond more or less to the extension of CS services and the introduction of PS services (data connectivity over GPRS / EDGE). The MAP protocol was extended in a backward-compatible manner between versions 1 and 2.

Things are getting more complicated with the development of the 3G standard by the 3GPP, the transfer of the MAP protocol maintenance to the 3GPP and its extension to version 3 and further. Starting with this version 3 of MAP, some backward-incompatible changes were introduced in the protocol: this means that some message definitions from MAPv3 cannot be used to encode or decode a MAPv1 or MAPv2 message. Here is an example of a MAP CancelLocation prototype taken from the open-source tool pycrate_map_op_info.py:

$ pycrate_map_op_info.py -o 3

--------------------------------------------------------------------------------
-------------------------  operationCode: (local, 03)  -------------------------
--------------------------------------------------------------------------------

MAP version 3 and over
OPERATION content: ArgumentType - Errors - ResultType - operationCode

  ArgumentType: CancelLocationArg (SEQUENCE)
    - identity (CHOICE)
    - cancellationType (ENUMERATED)
    - extensionContainer (SEQUENCE)
    - typeOfUpdate (ENUMERATED)
    - mtrf-SupportedAndAuthorized (NULL)
    - mtrf-SupportedAndNotAuthorized (NULL)
    - newMSC-Number (OCTET STRING)
    - newVLR-Number (OCTET STRING)
    - new-lmsi (OCTET STRING)
    - reattach-Required (NULL)
    mandatory : identity

  ResultType: CancelLocationRes (SEQUENCE)
    - extensionContainer (SEQUENCE)
    mandatory : 


MAP version 1 and 2
OPERATION content: ArgumentType - Errors - operationCode

  ArgumentType: CancelLocationArg (CHOICE)
    - imsi (OCTET STRING)
    - imsi-WithLMSI (SEQUENCE)


Initiator in MAP application context:
  - locationCancellationContext-v2           (0 4 0 0 1 0 2 2)
    {hlr} -> {sgsn, vlr}
  - locationCancellationContext-v1           (0 4 0 0 1 0 2 1)
    {hlr} -> {sgsn, vlr}
  - locationCancellationContext-v3           (0 4 0 0 1 0 2 3)
    {hlr} -> {sgsn, vlr}

Here, we can see that the ArgumentType of this operation was changed from a simple CHOICE to a potential OCTET STRING in version 1 and 2 to a more complex SEQUENCE with all sorts of information in version 3 and further. Those prototypes produce different encoding when MAP version 1 or 2 is used compared to when MAP version 3 is used. Let’s see how a simple IMSI encodes into such CancelLocationArg with Python3:

>>> from pycrate_asn1dir.TCAP_MAPv2v3 import *
[...]
>>> from pycrate_mobile.TS24008_IE import encode_bcd
>>> from binascii import hexlify
>>> IMSI = '208771234567890' 
>>>
>>> CLArg_v2 = MAPv2_MS_DataTypes.CancelLocationArg # MAP version 2
>>> CLArg_v2.set_val( ('imsi', encode_bcd(IMSI)) )
>>> print(CLArg_v2.to_asn1())
imsi : '02781732547698F0'H
>>> print(hexlify(CLArg_v2.to_ber()))
b'040802781732547698f0'
>>> 
>>> CLArg_v3 = MAP_MS_DataTypes.CancelLocationArg # MAP version 3
>>> CLArg_v3.set_val( {'identity': ('imsi', encode_bcd(IMSI))} )
>>> print(CLArg_v3.to_asn1())
{
  identity imsi : '02781732547698F0'H
}
>>> print(hexlify(CLArg_v3.to_ber()))
b'a30a040802781732547698f0'
>>>
>>> CLArg_v3.from_ber( CLArg_v2.to_ber() )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mich/python/pycrate_asn1rt/asnobj.py", line 1370, in from_ber
    self._from_ber(char, TLV)
  File "/home/mich/python/pycrate_asn1rt/asnobj.py", line 1347, in _from_ber
    .format(self.fullname(), (cl, pc, tval))))
pycrate_asn1rt.err.ASN1BERDecodeErr: CancelLocationArg: invalid tag class / pc / value, (0, 0, 4)

One can see that the version 2 structure encodes differently than the version 3: in version 3, there are 2 bytes 0xA30A prepended to the encoded buffer, which corresponds to the tag-length prefix of a SEQUENCE in BER. Moreover, we also note that the version 3 structure cannot decode a buffer encoded with the version 2 structure.

During the initial stage of a TCAP-MAP transaction, when a MAP version 2 (or later) operation is used, the MAP application-context has to be indicated (and eventually negotiated) into the DialoguePortion, that contains the MAP version to be used. At first sight, one could think that this application-context indication enables the receiver of such a message to know the MAP version and use the appropriate version to decode it.

This is however not true in many call flows, where a MAP operation is initiated only after the TCAP transaction is established and the application-context has been negotiated: in this case, the MAP ArgumentType will be in the ComponentPortion, but the DialoguePortion will be empty. This makes the decoding of TCAP-MAP message contextual (or stateful)!

This means that, in order to properly decode a TCAP-MAP message, one needs to know the MAP version that is used between the two TCAP endpoints, but this version is often only indicated in the first message of the TCAP transaction.

In order to decode TCAP-MAP messages in a stateless fashion (decoding a given message without knowing previous TCAP-MAP messages exchanged within a TCAP transaction), the decoder needs to try to decode the message for a given MAP version or another, and evaluate if the decoding succeeded or not. This impacts all equipment decoding TCAP-MAP messages without being active TCAP endpoints, such as STP, SS7 traffic analyzers and firewalls.

And the almighty Pycrate TCAP-MAP module

In order to deal with this specific issue, the Pycrate ASN.1 compiler and run-time were slightly extended to support table constraints with non-unique entries. This enables us to compile a special TCAP-MAPv2v3 ASN.1 module, in which both MAPv3 and MAPv2 message structures can be obtained for a single MAP operation code. The run-time then uses those structures to decode a given buffer until the decoding succeeds, and provides the correctly decoded value.

In addition to this specific handling of incompatibilities between MAP versions, this module integrates the MAP-DialoguePDU definition within the DialoguePortion part, and Ericsson and Nokia private MAP extensions available from the Wireshark project. Finally, a last enhancement consists of a custom-made ASN.1 module for MAP application-contexts, which defines MAP operation-packages and application-contexts together with the equipment responsible for those operations.

All those improvements over the original TCAP-MAP module enable to run a new tool within Pycrate: pycrate_map_op_info.py. This tool enables to search for a specific MAP operation code the corresponding prototype of ArgumentType, ResultType, defined error codes and ParameterType, depending on the MAP version, and to indicate also in which MAP application-context this operation can be initiated. Here is another example of this application with the UpdateLocation operation:

$ pycrate_map_op_info.py -o 2 -x -e

--------------------------------------------------------------------------------
-------------------------  operationCode: (local, 02)  -------------------------
--------------------------------------------------------------------------------

MAP version 3 and over
OPERATION content: ArgumentType - Errors - ResultType - operationCode

  ArgumentType: UpdateLocationArg (SEQUENCE)
    - imsi (OCTET STRING)
    - msc-Number (OCTET STRING)
    - vlr-Number (OCTET STRING)
    - lmsi (OCTET STRING)
    - extensionContainer (SEQUENCE)
    - vlr-Capability (SEQUENCE)
        {
        supportedCamelPhases: 'BIT STRING',
        extensionContainer: 'SEQUENCE',
        solsaSupportIndicator: 'NULL',
        istSupportIndicator: 'ENUMERATED',
        superChargerSupportedInServingNetworkEntity: ('CHOICE', {
         sendSubscriberData: 'NULL',
         subscriberDataStored: 'OCTET STRING'
         }),
        longFTN-Supported: 'NULL',
        supportedLCS-CapabilitySets: 'BIT STRING',
        offeredCamel4CSIs: 'BIT STRING',
        supportedRAT-TypesIndicator: 'BIT STRING',
        longGroupID-Supported: 'NULL',
        mtRoamingForwardingSupported: 'NULL',
        msisdn-lessOperation-Supported: 'NULL'
        }
    - informPreviousNetworkEntity (NULL)
    - cs-LCS-NotSupportedByUE (NULL)
    - v-gmlc-Address (OCTET STRING)
    - add-info (SEQUENCE)
        {
        imeisv: 'OCTET STRING',
        skipSubscriberDataUpdate: 'NULL'
        }
    - pagingArea (SEQUENCE OF)
        ('CHOICE', {
        laiFixedLength: 'OCTET STRING',
        lac: 'OCTET STRING'
        })
    - skipSubscriberDataUpdate (NULL)
    - restorationIndicator (NULL)
    - eplmn-List (SEQUENCE OF)
        'OCTET STRING'
    - mme-DiameterAddress (SEQUENCE)
        {
        diameter-Name: 'OCTET STRING',
        diameter-Realm: 'OCTET STRING'
        }
    mandatory : imsi, msc-Number, vlr-Number

  ResultType: UpdateLocationRes (SEQUENCE)
    - hlr-Number (OCTET STRING)
    - extensionContainer (SEQUENCE)
    - add-Capability (NULL)
    - pagingArea-Capability (NULL)
    mandatory : hlr-Number

  errorCode: (local, 34)
    ParameterType: SystemFailureParam (CHOICE)
      - networkResource (ENUMERATED)
      - extensibleSystemFailureParam (SEQUENCE)
            {
          networkResource: 'ENUMERATED',
          extensionContainer: 'SEQUENCE',
          additionalNetworkResource: 'ENUMERATED',
          failureCauseParam: 'ENUMERATED'
          }

  errorCode: (local, 35)
    ParameterType: DataMissingParam (SEQUENCE)
      - extensionContainer (SEQUENCE)
      mandatory : 

  errorCode: (local, 36)
    ParameterType: UnexpectedDataParam (SEQUENCE)
      - extensionContainer (SEQUENCE)
      - unexpectedSubscriber (NULL)
      mandatory : 

  errorCode: (local, 01)
    ParameterType: UnknownSubscriberParam (SEQUENCE)
      - extensionContainer (SEQUENCE)
      - unknownSubscriberDiagnostic (ENUMERATED)
      mandatory : 

  errorCode: (local, 08)
    ParameterType: RoamingNotAllowedParam (SEQUENCE)
      - roamingNotAllowedCause (ENUMERATED)
      - extensionContainer (SEQUENCE)
      - additionalRoamingNotAllowedCause (ENUMERATED)
      mandatory : roamingNotAllowedCause


MAP version 1 and 2
OPERATION content: ArgumentType - Errors - ResultType - operationCode

  ArgumentType: UpdateLocationArg (SEQUENCE)
    - imsi (OCTET STRING)
    - locationInfo (CHOICE)
        {
        roamingNumber: 'OCTET STRING',
        msc-Number: 'OCTET STRING'
        }
    - vlr-Number (OCTET STRING)
    - lmsi (OCTET STRING)
    mandatory : imsi, locationInfo, vlr-Number

  ResultType: UpdateLocationRes (CHOICE)
    - hlr-Number (OCTET STRING)
    - extensibleUpdateLocationRes (SEQUENCE)
        {
        hlr-Number: 'OCTET STRING'
        }

  errorCode: (local, 34)
    ParameterType: NetworkResource (ENUMERATED)

  errorCode: (local, 35)

  errorCode: (local, 36)

  errorCode: (local, 01)

  errorCode: (local, 08)
    ParameterType: RoamingNotAllowedCause (ENUMERATED)


Initiator in MAP application context:
  - networkLocUpContext-v1                   (0 4 0 0 1 0 1 1)
    {vlr} -> {hlr}
  - networkLocUpContext-v2                   (0 4 0 0 1 0 1 2)
    {vlr} -> {hlr}
  - networkLocUpContext-v3                   (0 4 0 0 1 0 1 3)
    {vlr} -> {hlr}

Using Pycrate, we are able to list all MAP operation codes that are used in both MAP version 1/2, and version 3 and over: 2, 3, 4, 7, 8, 10, 11, 12, 13, 14, 17, 18, 22, 29, 33, 34, 37, 43, 45, 46, 47, 50, 51, 55, 56, 57, 58, 59, 60, 61, 63, 64, 66, 67, 68 and 69. We are also able to find operations’ ArgumentType that are strongly incompatible between MAP version 1/2 and 3+, in addition to CancelLocation (operation code 3):

• operation code 43 (CheckIMEI) where an OCTET STRING is expected in version 1/2, whereas a SEQUENCE is expected in version 3+,
• operation code 56 (SendAuthenticationInfo) where again an OCTET STRING is expected in version 1/2, whereas a SEQUENCE is expected in version 3+.

One can find more incompatibilities by inspecting some ArgumentType that are SEQUENCE in all MAP versions, but the SEQUENCE’s content and / or the mandatory part is different. There are also similar incompatibilities into certain ResultType for responses and ParameterType for error indication.

Conclusion

TCAP-MAP is a complex protocol, that has evolved since the very early 1990s and continues to be extended today to support the evolution of mobile networks. Some backward incompatibilities were introduced when the protocol moved from version 2 to version 3.

Due to the way TCAP and MAP use ASN.1, with heavy parameterization, it is difficult to rely on this formal notation to build proper encoders and decoders ; this is however possible and a real advantage, compared to hand-crafted implementations that will suffer from issues and bugs in the support of many quirks imposed by the ASN.1 Basic Encoding Rules. Pycrate proposes a complete support of TCAP-MAP, for version 2, for version 3, for both versions within the same module, and also for TCAP-CAMEL. Do not hesitate to use them, report potential issues and contribute back to Pycrate on Github.

And, if interested in a job at P1 Security, do not hesitate to give a look at our open positions, such as software developer and lead developer.