FAQ

• What is the Log Level Data Service?
• What do I need to access the Log Level Data Service?
• What tools can I use to access data via the Log Level Data Service?
• How do I get logs into LiveRamp Safe Haven?
• How do I signup?
• When is data available for processing?
• How much does the Log Level Data Service cost?
• Who should I contact if something goes wrong?
• I am working with an attribution vendor who wants access to my data, can they use the Log Level Data Service?
• I can't see the S3 buckets when I login to my AWS console
• I get an error when trying to assume the IAM role from my account
• I am seeing Unicode errors when loading into Redshift
• How can I calculate the timestamp of an impression relative to the user?

What is the Log Level Data Service?

The Log Level Data Service is a MediaMath hosted data service that gives clients and partners an easy way to access raw, log-level data such as impressions or conversions with minimum of latency. Unlike traditional data feeds, the Log Level Data Service is a "pull" based model that leverages the cloud for scalable, reliable, and efficient data transfer.

What do I need to access the Log Level Data Service?

You will need a valid AWS account and an IAM user with sts:AssumeRole permissions. You will also need MediaMath to activate your account and assign a Role ARN. See Accessing Data Platform Using AWS for more details.

What tools can I use to access data via the Log Level Data Service?

The Log Level Data Service is designed to give you access to your raw, log-level data in a platform agnostic way. Any tool which can read data from Amazon S3 and process tab-delimited files will work. A few popular data solutions are listed below:

• Amazon Redshift
• Databricks
• Google Big Query
• Qubole
• Snowflake

How do I get logs into LiveRamp Safe Haven?

When you're working in Safe Haven, you can authorize LiveRamp's AWS user to retrieve data from another AWS s3, including those used for the MediaMath Log Level Data Service.

When requesting log level data access for LiveRamp on behalf of a brand or agency, include the following in your email to MediaMath Support:

• Be sure to mention “Log Level Service Setup”
• A list of MediaMath organization names and IDs to activate
• User ARN: arn:aws:iam::609251445204:user/svc-liveramp-ingestion-connectors
• Required Permissions:

ListBucket
GetBucketLocation
GetBucketPolicy
GetObject
ListBucketVersions

How do I signup?

To get started, send an email to MediaMath Support with the following information:

• Be sure to mention “Log Level Service Setup”
• Include a technical contact name and email
• 12-digit AWS account ID
• A list of MediaMath organization names and IDs to enable

We'll reply with a role ARN and external ID that can be used to access the log level data service.

If you want to allow a vendor or partner to access your log level data on your behalf, email them your approval so they can share that with us and we know it's ok to process their request.

When is data available for processing?

See Data Update Cycle

How much does the Log Level Data Service cost?

Please contact your account representative with questions on pricing.

Who should I contact if something goes wrong?

If you encounter any issues or have any questions, please contact MediaMath Support.

I am working with an attribution vendor who wants access to my data, can they use the Log Level Data Service?

Yes! The Log Level Data Service has a flexbile permissions model and your data can be made availble to partners on of your choosing. If you want to allow a vendor or partner to access your log level data on your behalf, email them your approval so they can share that with us and we know it's ok to process their request.

I can't see the S3 buckets when I login to my AWS console

The AWS console only displays S3 buckets that were created from within your account, not all the buckets that you have access to. You can use the AWS CLI tools to verify access to your buckets. For example:

$ aws s3 ls 2014-02-27 12:24:44 mm-prod-platform-attributed-events \
2013-11-26 11:39:52 mm-prod-platform-events \
2013-11-26 11:39:44 mm-prod-platform-impressions

See the Data Platform guide for more information.

I get an error when trying to assume the IAM role from my account

When accessing the Log Level Data Service, you will have to assume an IAM role. If you encounter an error such as the below

<ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">   
    <Error>     
        <Type>Sender</Type>     
        <Code>AccessDenied</Code>  
           <Message>User: arn:aws:iam::924339635232:user/xxxxxxxxx is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::794878508631:role/DPA-YYYYYYYYY</Message>  
    </Error> 
    <RequestId>75aad88e-bf26-11e3-890e-ad2fac39d231</RequestId> 
</ErrorResponse>

you must ensure that your IAM user has the privilege to assume a role in your account... You can apply the following policy to your user (or group)

{
  "Statement": [
    {
      "Action": [
        "sts:AssumeRole"
      ],
      "Effect": "Allow",
      "Resource": [
        "*"
      ],
      "Sid": "Stmt1389115902000"
    }
  ],
  "Version": "2012-10-17"
} 

I am seeing Unicode errors when loading into Redshift

If Redshift is giving you errors such as String contains invalid or unsupported UTF8 codepoints. Bad UTF8 hex sequence: 94 (error 3) you may have entered an invalid unicode character into TerminalOne when naming a campaign or strategy.

This is an easy fix though, just ensure that your COPY command replaces invalid characters with another character of your choosing, such as _or ?.

sql
    COPY your_table_name FROM 's3://mm-prod-data-platform/data/organization_id=<ORG_ID>'
    WITH CREDENTIALS
      'aws_access_key_id=<ACCESS_KEY>;aws_secret_access_key=<SECRET_KEY>;token=<TOKEN>'
    DELIMITER '\t'
    NULL AS '\\\\N'
    ACCEPTINVCHARS AS '?'
    LZOP
    TRUNCATECOLUMNS;

How can I calculate the timestamp of an impression relative to the user?

Data Platform logs provide a GMT timestamp for each record in each of the three datasets provided (impressions, events and attributed events). For impressions, a timestamp adjusted from GMT to the time zone of the campaign which served the impression is also provided, as this is the timestamp used for reporting purposes in TerminalOne. For attributed events, the timestamp of the event (pixel fire) is provided both in GMT and in the time zone of the campaign to which the event is attributed. Additionally the GMT timestamp of the impression to which the event is attributed in TerminalOne is provided. For event (pixel) logs, only the GMT timestamp of the pixel fire is provided.

On occasion a Data Platform user might be interested in the timestamp of an impression relative to the user who viewed the impression. While this is not provided explicitly, this can be deduced using the other fields in the Data Platform logs. The impression logs contain a field called WEEK_HOUR_PART, an integer between 0 and 671 representing a 15-minute interval in the week. For example:

The WEEK_HOUR_PART refers to the 15-minute interval relative to the timezone of the user. Thus, in general, the GMT timestamp of the impression will not match the WEEK_HOUR_PART value provided (the WEEK_HOUR_PART value is used by Brain for optimization purposes and therefore uses the user's timezone in determining what times of day/week users are particularly responsive or unresponsive to display media). But by calculating the 0-671 value of the given GMT timestamp and comparing to the WEEK_HOUR_PART value of the record, the user's GMT offset can be determined, and thus the timestamp of the impression relative to the user's time zone can be determined.

Suppose an impression is shown at five minutes after noon GMT on Tuesday, July 1, 2014 by a TerminalOne campaign with a configured time zone of America/New_York (UTC -04:00). The record for this impression will show a TIMESTAMP_GMT of '2014-07-01 12:05:00' and a REPORT_TIMESTAMP of '2014-07-01 08:05:00'. Suppose further that this impression was shown to a user in Berlin, Germany (UTC +02:00) and thus the WEEK_HOUR_PART in the log would be 248. Transforming the GMT timestamp of the impression would yield a 0-671 "15-minute interval" value of 240. Therefore the user who viwed the impression was 8 "intervals" ahead of GMT, and since each interval is 15 minutes, the user was in a time zone two hours ahead of GMT (i.e., UTC +02:00). Adding these two hours to the GMT timestamp of the impression yields a USER_TIMESTAMP of '2014-07-01 02:05:00'.