Showing posts with label wcf. Show all posts
Showing posts with label wcf. Show all posts

Wednesday, April 27, 2016

Erlang http response emulator

Recently I had to implement a "transparent" proxy between Tibco and Blackberry. I thought I'd share the erlang module I wrote to emulate responses from BB as you can't really expect from BB server to give you error responses or timeouts so you can test your proxy part. Here it is:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
-module(httpemulator).
-author("stanislavdvoychenko").

-export([client/0, server/0,start/0,accept/1,enter_loop/1,loop/1]).

client() ->
  {ok, Socket} =  gen_tcp:connect("localhost", 4001,[list, {packet, 0}]),
  ok = gen_tcp:send(Socket, "packet"),
  receive
    {tcp,Socket,String} ->
      io:format("Client received = ~p~n",[String]),
      io:format("Client result = ~p~n",[String]),
      gen_tcp:close(Socket)
  after 1000 ->
    exit
  end.

server() ->
  Pid = spawn(fun()-> start() end),
  Pid.

start() ->
  io:format("Started Server:~n"),
  {ok, Socket} = gen_tcp:listen(4001, [binary, {packet, 0},{reuseaddr, true},{active, false}]),
  accept(Socket).

accept(ListenSocket) ->
  io:format("Accept Server:~n"),
  case gen_tcp:accept(ListenSocket) of
    {ok, Socket} ->
      Pid = spawn(fun() ->
        io:format("Connection accepted ~n", []),
        enter_loop(Socket)
                  end),
      io:format("Pid ~p~n",[Pid]),
      gen_tcp:controlling_process(Socket, Pid),
      Pid ! ack,
      accept(ListenSocket);
    Error ->
      exit(Error)
  end.

enter_loop(Socket) ->
  %% make sure to acknowledge owner rights transmission finished
  receive ack -> ok end,
  loop(Socket).

loop(Socket) ->
  io:format("Loop Server:~n"),
  case gen_tcp:recv(Socket, 0) of
    {ok, Data} ->
          io:format("Server got data = ~s~n", [Data]),
          Match = re:run(Data, ".*Emulator-Command:\s(.+)\r", [{capture, [1],list}]),
      case Match of
        {match, ["timeout"]} -> Response = timeout("timeout");
        {match, ["protocolviolation"]} -> Response = protocol_violation("Omitting carriage return, this should cause the protocol violation");
        {match, ["httperror"]} -> Response = http_error("500", "Internal Server Error");
        _ -> Response = response("testing")
      end,

io:format("Match: ~p~n", [Match]),
          gen_tcp:send(Socket, Response);
          %%loop(Socket);
    {error, Reason} ->
      io:format("Error on socket ~p reason: ~p~n", [Socket, Reason]),
      gen_tcp:close(Socket)
  end.

response(Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [size(B), B])).

timeout(Str) ->
  timer:sleep(1000000000),
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [size(B), B])).

protocol_violation(Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\nContent-Type: text/html\nContent-Length: ~p\n\n~s",
      [size(B), B])).

http_error(Code, Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 ~s ~s\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [Code, B, size(B), B])).

This Erlang module expects you to sent an extra http header Emulator-Command: * where * can be:

httperror -> returns 500 and Internal Server Error
timeout -> runs a long sleep on Erlang side, causes timeout on a proxy side
protocolviolation -> As I built the proxy in ,net WCF it was funny to see that MS takes only \n in the http headers as a protocol violation, so \r\n is required to keep MS happy.

anything else there or omitting this header will just return proper http response with "testing" body.

Disclaimer. I'm only starting to learn Erlang so please take the code with caution of course!

To compile in the erlang shell:

c(httpemulator).

To start emulator server:

httpemulator:server().

To end the emulator process:

exit(pid(0,96,0), old).

Pid is returned at the server start.

Hope it can be useful :).

Friday, April 1, 2011

WCF: Memory leak with TypedMessageConverter when using XmlSerializer

Ok, hint is given in the title, but here is the code for you to observe and tell before you go any further what kind exactly of memory leak its causing and why :).

   1: private Message CreateResponseMessage(GetServiceStatusResponse result, Message message)
   2: {
   3:     TypedMessageConverter converter = TypedMessageConverter.Create(typeof(getServiceStatusResponse1), "*", "http://www.sitronics.com/V2/SCAdapter", new XmlSerializerFormatAttribute() );
   4:     
   5:     Message reply = converter.ToMessage(new getServiceStatusResponse1
   6:     {
   7:         GetServiceStatusResponse = result,
   8:         OutboundServiceData = new OutboundServiceData { MsgCorrelations = new OutboundServiceDataMsgCorrelations { CorrelationID = RouterService.CorrelationId } }
   9:     }, OperationContext.Current.IncomingMessageVersion);
  10:  
  11:  
  12:     RouterService.CopyRequestToReply(message, reply);
  13:  
  14:     return reply;
  15: }

Previous debugging points into classic assembly heap loader leak with XmlSerializer temp assemblies (here is the example for how to debug from Tess: http://blogs.msdn.com/b/tess/archive/2006/02/15/532804.aspx)

It is obvious that there should be XmlSerializer somewhere inside. Road to finding it starts with how the TypedMessageConverter is created:

   1: public static TypedMessageConverter Create(Type messageContract, string action, string defaultNamespace, XmlSerializerFormatAttribute formatterAttribute)
   2: {
   3:     if (messageContract == null)
   4:     {
   5:         throw DiagnosticUtility.ExceptionUtility.ThrowHelperError(new ArgumentNullException("messageContract"));
   6:     }
   7:     if (defaultNamespace == null)
   8:     {
   9:         defaultNamespace = "http://tempuri.org/";
  10:     }
  11:     return new XmlMessageConverter(GetOperationFormatter(messageContract, formatterAttribute, defaultNamespace, action));
  12: }
  13:  

I’ll cut the story saying that inside that routes to creation of SerializerGenerationContext to instantiate the required serializers:

   1: private XmlSerializer[] GenerateSerializers()
   2: {
   3:     List<XmlMembersMapping> list = new List<XmlMembersMapping>();
   4:     int[] numArray = new int[this.Mappings.Count];
   5:     for (int i = 0; i < this.Mappings.Count; i++)
   6:     {
   7:         XmlMembersMapping item = this.Mappings[i];
   8:         int index = list.IndexOf(item);
   9:         if (index < 0)
  10:         {
  11:             list.Add(item);
  12:             index = list.Count - 1;
  13:         }
  14:         numArray[i] = index;
  15:     }
  16:     XmlSerializer[] serializerArray = this.CreateSerializersFromMappings(list.ToArray(), this.type);
  17:     if (list.Count == this.Mappings.Count)
  18:     {
  19:         return serializerArray;
  20:     }
  21:     XmlSerializer[] serializerArray2 = new XmlSerializer[this.Mappings.Count];
  22:     for (int j = 0; j < this.Mappings.Count; j++)
  23:     {
  24:         serializerArray2[j] = serializerArray[numArray[j]];
  25:     }
  26:     return serializerArray2;
  27: }
  28:  

This “pre-cached” set of serializers is then used:

   1: internal XmlSerializer GetSerializer(int handle)
   2: {
   3:     if (handle < 0)
   4:     {
   5:         return null;
   6:     }
   7:     if (this.serializers == null)
   8:     {
   9:         lock (this.thisLock)
  10:         {
  11:             if (this.serializers == null)
  12:             {
  13:                 this.serializers = this.GenerateSerializers();
  14:             }
  15:         }
  16:     }
  17:     return this.serializers[handle];
  18: }
  19:  
  20:  

And just to complete the cycle of information, here is the inners of the XmlSerializer to return serializers from mappings:

   1: [PermissionSet(SecurityAction.LinkDemand, Name="FullTrust")]
   2: public static XmlSerializer[] FromMappings(XmlMapping[] mappings, Type type)
   3: {
   4:     if ((mappings == null) || (mappings.Length == 0))
   5:     {
   6:         return new XmlSerializer[0];
   7:     }
   8:     XmlSerializerImplementation contract = null;
   9:     Assembly assembly = (type == null) ? null : TempAssembly.LoadGeneratedAssembly(type, null, out contract);
  10:     TempAssembly tempAssembly = null;
  11:     if (assembly == null)
  12:     {
  13:         if (XmlMapping.IsShallow(mappings))
  14:         {
  15:             return new XmlSerializer[0];
  16:         }
  17:         if (type != null)
  18:         {
  19:             return GetSerializersFromCache(mappings, type);
  20:         }
  21:         tempAssembly = new TempAssembly(mappings, new Type[] { type }, null, null, null);
  22:         XmlSerializer[] serializerArray = new XmlSerializer[mappings.Length];
  23:         contract = tempAssembly.Contract;
  24:         for (int j = 0; j < serializerArray.Length; j++)
  25:         {
  26:             serializerArray[j] = (XmlSerializer) contract.TypedSerializers[mappings[j].Key];
  27:             serializerArray[j].SetTempAssembly(tempAssembly, mappings[j]);
  28:         }
  29:         return serializerArray;
  30:     }
  31:     XmlSerializer[] serializerArray2 = new XmlSerializer[mappings.Length];
  32:     for (int i = 0; i < serializerArray2.Length; i++)
  33:     {
  34:         serializerArray2[i] = (XmlSerializer) contract.TypedSerializers[mappings[i].Key];
  35:     }
  36:     return serializerArray2;
  37: }
  38:  
  39:  
  40:  
  41:  

So, now we can quite easily say what would be the problem of implementation on the top. Unmercifully, it leads to generating temp assemblies for the types in question every time it is called. Code snippets provided are to confirm and understand when/why that would happen or not.

So solution will be to “cache” the instance of TypedMessageConverter per type required, same as a usual solution when using XmlSerializer itself with “non-caching” constructors.

Thursday, February 17, 2011

WCF–maxSizeOfMessageToLog and "Message not logged because its size exceeds configured quota"

I’m always in for investigating some features that smell of a bad design. And though I believe WCF is a very solid piece of a framework, this warning message while skipping the message log makes me nervous. By itself surely it is better to have an ability to skip logging messages over threshold size, but if I made my conscious decision through configuration for that size, why should I still be given a warning?
I’m used to log warnings and up into EventLog for admins to be alerted. This warning just makes my admins alerted for nothing. Not good.

So I wanted to see if I can tame this in some legitimate way. I started with msdn:

http://msdn.microsoft.com/en-us/library/system.servicemodel.configuration.messageloggingelement.maxsizeofmessagetolog.aspx

Which by the way states about maxSizeOfMessageToLog the following: "The maximum size, in bytes, of a message to log. Messages larger than the limit are not logged. This setting affects all trace levels. The default is Int32.MaxValue"

Int32.MaxValue??!! This would not be corresponding to any “secure by default” and to the real life experience working with wcf! So dive into the reflector shows:

 

internal MessageLogTraceRecord(Stream stream, MessageLoggingSource source) : this(source)
{
    this.type = null;
    StringBuilder builder = new StringBuilder();
    StreamReader reader = new StreamReader(stream);
    int size = 0x1000;
    char[] buffer = DiagnosticUtility.Utility.AllocateCharArray(size);
    int maxMessageSize = MessageLogger.MaxMessageSize;
    if (-1 == maxMessageSize)
    {
        maxMessageSize = 0x1000;
    }
    while (maxMessageSize > 0)
    {
        int num3 = reader.Read(buffer, 0, size);
        if (num3 == 0)
        {
            break;
        }
        int charCount = (maxMessageSize < num3) ? maxMessageSize : num3;
        builder.Append(buffer, 0, charCount);
        maxMessageSize -= num3;
    }
    reader.Close();
    this.messageString = builder.ToString();
}

As we see above the origins of the default value are found in the MessageLogTraceRecord with its value of 4096 bytes as per above (which is quite reasonable I think).

As it often happens bad smells are coming in batches, so we can see in TraceXPathNavigator which is used by PlainXmlWriter (all in System.ServiceModel?.Diagnostics namespace) how “business” case of exceeding the threshold is handled via exception:

private void VerifySize(int nodeSize)
{   
  if ((this.maxSize != -1) && ((this.currentSize + nodeSize) > this.maxSize))  
  {      
    throw new PlainXmlWriter.MaxSizeExceededException();    
  }  
    this.currentSize += nodeSize;
  }

 

And now to complete the parade this exception is handled by creation of a Warning message in System.ServiceModel.Diagnostics.MessageLogger LogInternal method:

catch (PlainXmlWriter.MaxSizeExceededException)
    {
        if (DiagnosticUtility.ShouldTraceWarning)
        {
            TraceUtility.TraceEvent(TraceEventType.Warning, TraceCode.MessageNotLoggedQuotaExceeded, record.Message);
        }
    }

This is quite far from any good design choice in my opinion! By bet is that this guy has moved from WCF team to Silverlight’s already. SmileSmile But before he did that, he made it sure we have no legitimate way to avoid that warning message and probably didn’t tell the truth to the msisdn documentation team.

Friday, December 10, 2010

WCF: TypedMessageConverter and its default serializer

At some point you may face a need to create a Message from your MessageContract.
The helper class for this is TypedMessageConverter with possible usage as:

TypedMessageConverter converter = TypedMessageConverter.Create(
 typeof(getServiceStatusResponse1), "", "YourDefaultNamespace")
Message reply = converter.ToMessage(…..

Next second, when looking at the message generated, you may see quite odd looking result with all the backing fields exposed instead of your properties and set off namespaces. What’s the heck???

If we look behind one of create overloads:

 
public static TypedMessageConverter Create(Type messageContract, string action) {
 
    return Create(messageContract, action, null, 
              TypeLoader.DefaultDataContractFormatAttribute); 
 
} 

Where TypeLoader has the following DefaultDataContractAttribute:

internal static DataContractFormatAttribute DefaultDataContractFormatAttribute;

This in turn commands it to use a DataContractSerializer and screw Yours very carefully designed XmlType/Element etc annotations.

The cure is very simple, create a typed converter with XmlSerializerFormatAttribute:

TypedMessageConverter converter = TypedMessageConverter.Create(typeof(getServiceStatusResponse1),
 "", "YourDefaultNamespace", new XmlSerializerFormatAttribute() ); 

And feel the joy of Your perfectly looking Message!

[Update 1-Apr-2011, you can read http://plainoldstan.blogspot.com/2011/04/wcf-memory-leak-with.html to avoid possible memory leaks when you use TypedMessageConverter with XmlSerializer]