MATE's configuration tutorial

If you are new to MATE, you should first read the Getting Started page.

We'll show a MATE configuration that first creates Gops for every DNS and HTTP request, then it ties the Gops together in a Gop based on the host. Finally we'll separate into different Gogs request coming from different users.

With this MATE configuration loaded we can:

The complete config file is here: web.mate

Note: for this example I used dns.qry.name which is defined since Wireshark version 0.10.9. Supposing you have a mate plugin already installed you can test it with the current Wireshark version.

A Gop for DNS requests

First we'll tell MATE how to create a Gop for each DNS request/response.

MATE needs to know what makes a DNS PDU we do this with an Action=PduDef. 

Action=PduDef; Name=dns_pdu; Proto=dns; Transport=ip; addr=ip.addr; dns_id=dns.id; dns_resp=dns.flags.response;

Using Proto=dns we tell MATE to create Pdus every time it finds dns. With Transport=ip we inform MATE that some of the fields we are interested are in the ip part of the frame. Finally we tell MATE to import ip.addr as addr, dns.id as dns_id and dns.flags.response as dns_resp.

Once we've told MATE how to extract dns_pdus we'll tell MATE how to match requests and responses and get them into a Gop. For this we'll use Action=GopDef to define the Gop, and then, Action=GopStart and Action=GopStop to tell it when the Gop starts and ends. 

Action=GopDef; Name=dns_req; On=dns_pdu; addr; addr; dns_id;
Action=GopStart; For=dns_req; dns_resp=0;
Action=GopStop; For=dns_req; dns_resp=1;

In the Action=GopDef we tell MATE that the Name of the Gop is dns_req. Other than that we instruct MATE to check every dns_pdu for a match of the key.

The key for the Gop is "addr; addr; dns_id;" that means that in order to belong to the same Gop dns_pdus have to have both addresses and the request id identical. We then instruct MATE that a dns_req starts whenever a dns_pdu matches "dns_resp=0;". And, that it stops when another dns_pdu matches "dns_resp=1;".

At this point if we open a capture file using this configuration we are able to filter with mate.dns_req.Time > 1 to see only the packets of DNS requests that take more than one second to complete.

We can filter with mate.dns_req && ! mate.dns_req.Time to find requests for which no response was given. mate.xxx.Time is set only for Gops that have being stopped.

A Gop for HTTP requests

this other example creates a Gop for every HTTP request 

Action=PduDef; Name=http_pdu; Proto=http; DiscardPduData=TRUE; Transport=tcp/ip; addr=ip.addr; port=tcp.port; http_rq=http.request.method; http_rs=http.response;

Action=GopDef; Name=http_req; On=http_pdu; addr; addr; port; port;
Action=GopStart; For=http_req; http_rq;
Action=GopStop; For=http_req; http_rs;

So, if we open a capture using this configuration

You have to know that: mate.xxx.Time gives the time in seconds between the pdu matching the GopStart and the Pdu matching the GopStop (Yes, you can create timers using this!). On the other hand mate.xxx.Duration gives you the time passed between the GopStart and the last pdu assigned to that Gop regardless whether it is a stop or not. After the GopStop Pdus matching the Gop's Key will still be assigned to the same Gop as far as they don't match the GopStart in which case a new Gop with the same key will be created.

Getting DNS and HTTP together into a Gog

Using the Pdu and Gop definitions of the previous examples we'll tie together in a single Gog all the http packets belonging to requests and responses to a certain host and the dns request and response used to resolve its domain name.

To be able to group DNS and HTTP requests we need to import into the Pdus and Gops some part of information that both protocols share. Once the Pdus And Gops have being defined we can use Action=PduExtra and Action=GopExtra to tell MATE what other things are to be added to Pdus and Gops. 

Action=PduExtra; For=http_pdu; host=http.host; 
Action=GopExtra; For=http_req; host;

Action=PduExtra; For=dns_pdu; host=dns.qry.name;
Action=GopExtra; For=dns_req; host;

Here we've told MATE to import http.host into http_pdus and dns.qry.name into dns_pdu as host. Other than that we have to tell MATE to copy the host attribute from the Pdus to the Gops, we do this using GopExtra.

Once we got all the data we need in Pdus and Gops we tell MATE what makes different Gops belong to a certain Gog. 

Action=GogDef; Name=http_use; GogExpiration=0.75;
Action=GogKey; For=http_use; On=http_req; host;
Action=GogKey; For=http_use; On=dns_req; host;

With Action=GogDef we tell MATE to define a Gog type Named http_use whose expiration is 0.75 seconds after all the Gops that belong to it had been stopped. After that a new Gop with the same key match will create a new Gog instead of been added to the previous Gog.

With Action=GogKey we tell MATE that http_reqs with the same host belong to the same Gog, same thing for dns_reqs.

So far we have instructed mate to group every packet related to sessions towards a certain host. At this point if we open a capture file and:

Separating requests from multiple users

"Houston: we've had a problem here."

At this point requests from many users get mixed together into http_uses. Gogs are created and stopped almost randomly (depending on the timing in which Gops start and stop), this configuration works fine if used for captures form the clients's side but in the server side we've got a real mess. How do we get them separated?

MATE has a tool that can be used to resolve this kind of grouping issues, they are the Transforms. Once defined they can be applied against Pdus, Gops and Gogs and they might replace or insert more attributes based on what's there. We'll use them to create a client attribute with which we'll separate different requests.

For DNS we need the ip.src of the request moved into the Gop only for the request.

We tell MATE to import ip.src as client; 

Action=PduExtra; For=dns_pdu; client=ip.src;

We then tell MATE to replace in the Pdu "dns_resp=1; client;" with just "dns_resp=1;". That way we'll have the attribute client only on non-request Pdus (i.e. packets coming from the client). 

Action=Transform; Name=rm_client_from_dns_resp; Mode=Replace; dns_resp=1; client; .dns_resp=1;
Action=PduTransform; For=dns_pdu; Name=rm_client_from_dns_resp;

First we define the Transform (In this case with just one clause), then we apply it to the Pdu with PduTransform.

HTTP is a little trickier, we have to remove the attribute carrying ip.src from both the response and the "continuations" of the response, not having anything on which to filter for the continuations we'll have to add a fake attribute first. And then we'll have to remove client when the fake attribute appears. 

Action=PduExtra; For=http_pdu; client=ip.src;

# first match wins so the request won't get the not_rq attribute inserted
Action=Transform; Name=rm_client_from_http_resp1; Mode=Insert; Match=Strict; http_rq;

# this line won't be evaluated if the first one matched so not_rq won't be inserted to requests 
Action=Transform; Name=rm_client_from_http_resp1; Mode=Insert; Match=Every; addr; .not_rq;


# then try to replace "client and not_rq" with nothing (will happen only in the response and parts of it)
Action=Transform; Name=rm_client_from_http_resp2; Mode=Replace; Match=Strict; not_rq; client;

Action=PduTransform; For=http_pdu; Name=rm_client_from_http_resp1;
Action=PduTransform; For=http_pdu; Name=rm_client_from_http_resp2;

In MATE all the transforms declared for an item will be evaluated; While in a single transform evaluation will stop at the first successful match. That's why we first just matched http_rq, to get out of the first sequence before adding the not_rq attribute. Then we applied the second transform which removed both not_rq and client when both were there. Yes, Transforms are cumbersome, but they are very useful. The plan is to improve them, Any Ideas?

Once we got all what we need in the Pdus we have to tell MATE to copy the attribute client from the Pdus to the Gops. 

Action=GopExtra; For=dns_req; client;
Action=GopExtra; For=http_req; client;

Other than that we'll need to modyfy the old GopKey definitions to a new definition that matches both client and host. So we'll redefine the Gog as: 

Action=GogDef; Name=http_use; GogExpiration=0.75;
Action=GogKey; For=http_use; On=http_req; host; client;
Action=GogKey; For=http_use; On=dns_req; host; client;

Now we got it, every "usage" gets it's own Gog.

Mate/Tutorial (last edited 2008-06-19 18:54:01 by GuyHarris)