Pulling CDRs directly from PostgreSQL is not always a viable solution for our users.

With the Thrift feature enabled, the system processes CDRs normally, but you can configure an aggressive CDR purge cycle (two days or so).

The advantage of using Thrift is that you are empowered to ingest, process, and analyze your CDRs autonomously.


Below is a request made by one of our users that prompted our extension of the Thrift functionality with the softswitch.


Request:

As an operator of a wholesale switch, I want to retrieve CDRs in simple text format from my switch so that I can ingest that data into other systems for external use.

The switch must write Account and Vendor CDRs in an agreed text-based format (JSON or CSV) to flat files.

  • The files must be rotated every N minutes.
  • The files must be removed in a timely manner to ensure servers do not run out of space

File Naming Convention, format:

client_<time_from_epoch>_<rotation_frequency_in_seconds>
If the rotation time is set to 3600 seconds, and the file is rotated at Wed, 11 Dec 2013 22:02:15 GMT, then the file name will be 1386799335
client_1386799335_3600.bin


Additionally, thrift files can be uploaded to an SFTP server, and data can be parsed into the desired format with any 3rd party tool.


Main files:

  • balance_updates-thrift.bin - thrift file with data regarding balance updates for Account/Customer/Vendor that took part in the call
  • calls-thrift.bin - basic data for the call. It contains unique i_call record, which is used as a key to connect data from other tables 
  • cdrs-thrift.bin - Account's CDRs data
  • cdrs_connections-thrift.bin - Vendor's CDRs data


Optional files:

  • cdrs_customers-thrift.bin - Customer's CDRs data
  • cdrs_dids-thrift.bin - DID's CDRs data
  • cdrs_customers_dids-thrift.bin - delegated DID's CDRs data
  • cdrs_connections_dids-thrift.bin - Vendor's CDRs data for DID for the case Buying DID Charging Group is configured for DID Authentication rule
  • commissions-thrift.bin - CDRs data with extra commissions applies for the call
  • calls_sdp-thrift.bin - CDRs data with received SDP records, file would be present if Record SDP is enabled for an Environment
  • surcharges - CDRs data with extra charges.


Folder Path for Thrift Files:

 ttypes.py - Python classes generated from Thrift definition codes. python version in the path might vary for different releases.

/usr/local/lib/python3.9/site-packages/sippyapi/thrift/ttypes.py

 Thrift files - Thrift definition codes for classes that are used to generate code in your target languages and are stored in the following path:

/usr/local/share/ssp/

 

Custom Types for Thrift Files:

 Apart from the provided basic types, There are several custom types defined.

 For information about the basic types, visit the official Apache Thrift documentation.


types.thrift - Most basic custom types such as NullInt64, MonoTime, etc. are defined to use them as types for the Thrift struct. 

/usr/local/share/ssp/types.thrift

*Note: Some custom types, such as the enum type for Thrift struct, are defined in other relevant thrift files.


How to Generate Code for Your Target languages:

All code is defined in Python. However, Some customers might want to use classes in different languages. In order to generate code in your target languages, run the following command:

thrift --gen <language> <Thrift filename>

e.g. To generate code for ruby, run

thrift -r --gen rb types.thrift

For the further instructions and acronyms for languages, please refer to the documentation.


How to Prepare classes for calls-thrift.bin:

The main class is Calls, which is a return object for calls-thrift.bin (see the return data in the following EXAMPLE FROM SERVER). 


1. Create an empty thrift file in your current dir. Run the following command to create all.thrift file.

touch ./all.thrift


2. Check what types are used for Calls struct (struct name in .thrift files will be the same as class name in target

    languages. e.g. struct Calls{} will be converted to class Calls{}. )

   

  2.1.  The Calls struct includes the NullInt64 custom type, which is defined in types.thrift.


  2.2.  Copy the Calls struct from call_details.thrift and the NullInt64 custom type from types.thrift to the all.thrift file. The easiest way to handle that should be to check all struct definitions inside calls_details.thrift and identify all 

 custom types. In call_details.thrift, There is a LRN_RESULT, which is defined in lrn_cache.thrift. You can do the same process by running the following commands one by one to copy all the necessary data into all.thrift.

cat /usr/local/share/ssp/types.thrift > ./all.thrift
cat /usr/local/share/ssp/lrn_cache.thrift >> ./all.thrift
cat /usr/local/share/ssp/call_details.thrift >> ./all.thrift


 2.3. For the example, we'll generate Python-based code.

thrift -r --gen py ./all.thrift

You'll see a generated folder called gen-<acronyms for languages>. py was specified as a target language; therefore, the folder name should be gen-py. All necessary files are included in there.

All you have to do is create similar logic to thrift_parser.py and work on parsing binary data as shown in the following section.

 

EXAMPLE FROM SERVER:
*Note: thrift_parser.py only works for Python code. In order to parse binary files properly into your target languages, The equivalent parser code must be prepared beforehand.
$ /home/ssp/scripts/support_tools/thrift_parser.py /var/env1/upload/cdrs/thrift/calls-thrift.bin
Calls(local_i_call=6101993, call_id='3-57129@192.168.1.34', cld='12064248284', cli='Tenson Hokey', setup_time=1672226825, parent_local_i_call=None, i_call_type=None, i_routing_group=354, node_id='node-', i_call=935442)

$ /home/ssp/scripts/support_tools/thrift_parser.py /var/env1/upload/cdrs/thrift/cdrs-thrift.bin
Cdrs(i_cdr=2554692, i_account=34, result=0, cost=0.3603, delay=30.0, duration=2.183015525, billed_duration=2.0, connect_time=1672226855, disconnect_time=1672226857, cld_in='testnumber_12064248284', cli_in='sipp', prefix='1', price_1=0.0063, price_n=0.0043, interval_1=1, interval_n=1, post_call_surcharge=0.2, connect_fee=0.3, free_seconds=0, remote_ip='192.168.1.34', grace_period=0, user_agent='', pdd1xx=0.621326224, i_protocol=1, release_source='', plan_duration=0.0, accessibility_cost=0.0, lrn_cld=None, lrn_cld_in=None, area_name=None, p_asserted_id=None, remote_party_id=None, conn_proc_time=0.061621094, lrn_cli_in=None, lrn_cli=None, media_timeout_correction=0.0, lrn_cli_result=4, lrn_cld_result=4, i_call=935442)

$ /home/ssp/scripts/support_tools/thrift_parser.py /var/env1/upload/cdrs/thrift/cdrs_customers-thrift.bin
CdrsCustomers(i_cdrs_customer=885332, i_cdr=2554692, i_customer=31, cost=-0.3603, billed_duration=2.0, prefix='1', price_1=0.0063, price_n=0.0043, interval_1=1, interval_n=1, post_call_surcharge=0.2, connect_fee=0.3, free_seconds=0, grace_period=0, i_wholesaler=1, setup_time=1672226825, duration=2.183015525, area_name=None, media_timeout_correction=0.0, i_call=935442)

$ /home/ssp/scripts/support_tools/thrift_parser.py /var/env1/upload/cdrs/thrift/commissions-thrift.bin
Commissions(i_commission=753889, i_account=NullInt64(v=34), i_customer=None, i_cdrs_customer=885332, commission_size=1.0, setup_time=1672226825, i_call=935442)

$ /home/ssp/scripts/support_tools/thrift_parser.py /var/env1/upload/cdrs/thrift/calls_sdp-thrift.bin
CallsSdp(i_calls_sdp=4776912, i_cdrs_connection=None, time_stamp=UnixTime(seconds=1672226824, nanoseconds=583246946), sdp='v=0\r\no=user1 53655765 2353687637 IN IP4 192.168.1.34\r\ns=-\r\nc=IN IP4 192.168.1.34\r\nt=0 0\r\nm=audio 6000 RTP/AVP 0\r\na=rtpmap:0 PCMU/8000\r\n', sip_msg_type='INVITE', direction='received from', call_leg='customer', i_call=935442)

$ /home/ssp/scripts/support_tools/thrift_parser.py /var/env1/upload/cdrs/thrift/cdrs_connections-thrift.bin
CdrsConnections(i_cdrs_connection=2717960, i_connection=146, result=0, cost=0.001743333333333333, delay=0.0, duration=2.183015525, billed_duration=2.0, setup_time=1672226854, connect_time=1672226854, disconnect_time=1672226857, cld_out='+B003_translated_cld_12064248284', cli_out='Clever guy', prefix='1206', price_1=0.08, price_n=0.0246, interval_1=1, interval_n=1, post_call_surcharge=0.0, connect_fee=0.0, free_seconds=0, grace_period=0, user_agent='', pdd100=0.16096918600669596, pdd1xx=0.16096918600669596, i_account_debug=34, i_proto_transport=1, release_source='', call_setup_time=1672226824, lrn_cld=None, area_name=None, i_media_relay=NullInt64(v=20), remote_ip=NullString(s='vendor-name.provider.com:6070'), vendor_name=NullString(s='sipp_uas'), i_media_relay_outcome=NullInt64(v=1), media_ip='192.168.3.65', lrn_cli=None, call_id='3-57129@192.168.1.34-b2b_4', media_timeout_correction=0.0, huntstop=False, i_call=935442)