Wednesday, April 27, 2016

Erlang http response emulator

Recently I had to implement a "transparent" proxy between Tibco and Blackberry. I thought I'd share the erlang module I wrote to emulate responses from BB as you can't really expect from BB server to give you error responses or timeouts so you can test your proxy part. Here it is:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
-module(httpemulator).
-author("stanislavdvoychenko").

-export([client/0, server/0,start/0,accept/1,enter_loop/1,loop/1]).

client() ->
  {ok, Socket} =  gen_tcp:connect("localhost", 4001,[list, {packet, 0}]),
  ok = gen_tcp:send(Socket, "packet"),
  receive
    {tcp,Socket,String} ->
      io:format("Client received = ~p~n",[String]),
      io:format("Client result = ~p~n",[String]),
      gen_tcp:close(Socket)
  after 1000 ->
    exit
  end.

server() ->
  Pid = spawn(fun()-> start() end),
  Pid.

start() ->
  io:format("Started Server:~n"),
  {ok, Socket} = gen_tcp:listen(4001, [binary, {packet, 0},{reuseaddr, true},{active, false}]),
  accept(Socket).

accept(ListenSocket) ->
  io:format("Accept Server:~n"),
  case gen_tcp:accept(ListenSocket) of
    {ok, Socket} ->
      Pid = spawn(fun() ->
        io:format("Connection accepted ~n", []),
        enter_loop(Socket)
                  end),
      io:format("Pid ~p~n",[Pid]),
      gen_tcp:controlling_process(Socket, Pid),
      Pid ! ack,
      accept(ListenSocket);
    Error ->
      exit(Error)
  end.

enter_loop(Socket) ->
  %% make sure to acknowledge owner rights transmission finished
  receive ack -> ok end,
  loop(Socket).

loop(Socket) ->
  io:format("Loop Server:~n"),
  case gen_tcp:recv(Socket, 0) of
    {ok, Data} ->
          io:format("Server got data = ~s~n", [Data]),
          Match = re:run(Data, ".*Emulator-Command:\s(.+)\r", [{capture, [1],list}]),
      case Match of
        {match, ["timeout"]} -> Response = timeout("timeout");
        {match, ["protocolviolation"]} -> Response = protocol_violation("Omitting carriage return, this should cause the protocol violation");
        {match, ["httperror"]} -> Response = http_error("500", "Internal Server Error");
        _ -> Response = response("testing")
      end,

io:format("Match: ~p~n", [Match]),
          gen_tcp:send(Socket, Response);
          %%loop(Socket);
    {error, Reason} ->
      io:format("Error on socket ~p reason: ~p~n", [Socket, Reason]),
      gen_tcp:close(Socket)
  end.

response(Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [size(B), B])).

timeout(Str) ->
  timer:sleep(1000000000),
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [size(B), B])).

protocol_violation(Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 200 OK\nContent-Type: text/html\nContent-Length: ~p\n\n~s",
      [size(B), B])).

http_error(Code, Str) ->
  B = iolist_to_binary(Str),
  iolist_to_binary(
    io_lib:fwrite(
      "HTTP/1.0 ~s ~s\r\nContent-Type: text/html\r\nContent-Length: ~p\r\n\r\n~s",
      [Code, B, size(B), B])).

This Erlang module expects you to sent an extra http header Emulator-Command: * where * can be:

httperror -> returns 500 and Internal Server Error
timeout -> runs a long sleep on Erlang side, causes timeout on a proxy side
protocolviolation -> As I built the proxy in ,net WCF it was funny to see that MS takes only \n in the http headers as a protocol violation, so \r\n is required to keep MS happy.

anything else there or omitting this header will just return proper http response with "testing" body.

Disclaimer. I'm only starting to learn Erlang so please take the code with caution of course!

To compile in the erlang shell:

c(httpemulator).

To start emulator server:

httpemulator:server().

To end the emulator process:

exit(pid(0,96,0), old).

Pid is returned at the server start.

Hope it can be useful :).

Thursday, April 14, 2016

xcode - Configure for analyzing. The scheme is not configured for analyzing.

I'm used to run analyzing cycle on my apps when moving closer to their public release. This time xcode (7.3 (7D175)) greeted me with:


"Edit the scheme to enable analyzing, or cancel the action." with that "Edit Scheme ..." was not opening anything and I didn't notice anything extra in the schema editor to enable analyzing.

I even created a clean new project to see if error would be present there as well, and it was.

So after scratching my head a bit I just set a RUN_CLANG_STATIC_ANALYZER as true for the target/build where I needed it:


This way I got the static analysis messages back.

Hope this can help someone!

Update. Or as Jaime Santana commented, just "launch Analysis from the menu option Menu -> Product -> Analyze". Thank you Jamie, that worked!

Thursday, February 18, 2016

Note to self - concatenating rows into string with MSSQL

Here is a sample code:

select count(a.v2_icid), a.v2_icid,
(select distinct(fullname) + ',' AS [text()] from systemuser u 
join account acc on acc.ownerid = u.systemuserid 
where acc.v2_icid = a.v2_icid For XML PATH ('')) [Users]
from account a
where statecode = 0 and a.v2_icid is not null
group by a.v2_icid
having count(a.v2_icid) > 1
order by a.v2_icid

Monday, December 7, 2015

So, here is my string dump for x64 in windbg

Based on this and that.

$$>a<"c:\_install\dumpstringtofolder.txt" 000007feed816500 1000 c:\temp\stringtest

and the script:

$$ Dumps the managed strings to a file
$$ Platform x64
$$ First argument is the string method table pointer
$$ Second argument is the Min size of the string that needs to be used filter
$$ the strings
$$ Third is the path of the file
.foreach ($string {!dumpheap -short -mt ${$arg1}  -min ${$arg2}})
{
r@$t0=  poi(${$string}+8)*2
.writemem ${$arg3}${$string}.txt ${$string}+c L? @$t0
}

The only other thing I needed this time is this on one of the strings:

!gcroot 0000000100e769f0

To see:

   ->  0000000180006d08 System.Web.Caching.CacheMultiple
            ->  00000001800069c8 System.Web.Caching.CacheCommon
            ->  00000001800034a8 System.Web.RequestTimeoutManager
....
            ->  00000004021b5740 System.Web.SessionState.InProcSessionState
            ->  00000004021b5288 System.Web.SessionState.SessionStateItemCollection
....
            ->  0000000289779a58 ASP.ordercomposer_orderedit_aspx
            ->  000000018c14fe38 XYZ.Ordering.Modules.OrderingViewStatePresister

So this time the root of the problem was in keeping viewstate in the session and not really purging not needed view state away. This was one of the legacy apps and reminded everyone the old none-tasty view state solution from MS :). Viva MVC and looking forward the day with no legacy ASP.NET apps we need to support :)!

Tuesday, May 20, 2014

NGINX + PostgreSQL + PostGIS - performance baseline/diff

Just wanted to share some of the initial benchmark from http_load on the project I'm working right now. Why the excitement? I've been an MS oriented guy for the last 20 years and moving away from IIS and MSSQL (where you can be waiting 20-30 seconds for the initial IIS/.NET/SQL app startup) to NGINX with PostgreSQL stack is life changing - no exaggeration!

So here is what I got on Digital Ocean lowest spec machine (1 core, 20GB SSD, 512 MB RAM, Ubuntu 12) with a bare NGINX:

34178 fetches, 10 max parallel, 683560 bytes, in 5 seconds
20 mean bytes/connection
6835.6 fetches/sec, 136712 bytes/sec
msecs/connect: 0.0445469 mean, 0.937 max, 0.021 min
msecs/first-response: 1.41242 mean, 101.975 max, 0.257 min

And with the full nginx->postgresql with postgis searching for speed limits around submitted location (but only a few limits present in db for a moment):

5574 fetches, 10 max parallel, 6.22058e+06 bytes, in 5 seconds
1116 mean bytes/connection
1114.8 fetches/sec, 1.24412e+06 bytes/sec
msecs/connect: 0.0561055 mean, 0.37 max, 0.023 min
msecs/first-response: 8.68285 mean, 187.733 max, 0.812 min

So it got from almost 7k to 1k per second, still that's from the different universe compared to what IIS/.NET would do for me, and look at the machine spec! Plus, $0.0 for nginx + $0.0 for posgre/postgis - pure win!

-- Stan.

Friday, November 29, 2013

Apple MKMapView vs. Google iOS SDK map - CPU, compass image, center and rotate.

Why I didn't use native Apple's map a year ago when I was going through the options for my apps?

I was looking for the "live" map that would be able to follow your location, option to rotate so your heading can be always on top of the map and while it rotates it should rotate street names gracefully so they are not shown upside down.

Should I tell Apple maps for iOS6 failed miserably for all of those requirements. So I used Google's alternative. It gave me great rotation, street names handling and nothing to complain about.




Watching iOS7 location videos made me really intrigued, it looked like "oh, Apple fixed it all and brought us the map that can rotate correctly". I was really keen to try and after two months of fixing my apps for iOS7 and releasing them I finally got to try.... And I'm not sure this is it.

Why?

You basically have an option to either center the map yourself or set it up to track by itself:


[self.map setUserTrackingMode:[MKUserTrackingModeFollow|MKUserTrackingModeFollowWithHeading] animated:[true|false]

Well, there are few problems here. 

Rotate and center sync. If you want to rotate with one of the automated tracking options, you'd probably need to sync your rotation with the user location tracking animation. Otherwise, for my naive and short attempts the rotation and auto-tracking create weird effect of user location floating around its center. I didn't follow the path of trying to sync that yet and will see if this is worth an effort.

Compass image. Small extra annoying factor here is that once you rotate the map (set the map camera heading) to something not equal zero, the "compass" thingy appears in the right upper corner:



My first thought was - "oh, I'l move this thingy where I want or disable it for iPad version as I show heading elsewhere". Let me do a read up, let me google for it, let me study Apple's forum for it... What? I can't do it! I love Apple for things that were decided for us, it is part of their spirit and I'm glad they keep it even stronger now than before!

This compass thingy is not a big issue I can move my star button (that happened to be in this place by accident) - I'm not that picky.

It has glitches. From time to time it looks like map zoom is being reset for a moment and than re-established again. I can't really correlate it to whatever I'm doing (as I'm doing nothing but set camera's heading and I do for every location change while zoom "blink" only happens once in a longer while). And the first time I rotate it it goes through the field of black and white rectangular and then does it from time to time, probably when tiles has not been loaded yet for some area I suddenly need to expose by programmatic rotation. This never happens to the Google map.
+ When changing programmatically (or by setting the auto option with WithHeading) heading from 360 to 1, Apple's map rotates -360 + 1 degree. Like, this can't be true? I'm probably just silly. But Google does it right, and it makes me feel less silly and more desperate.
+ Compass image shows even when I have enableRotate to NO, but when phone points exactly to 0 heading, the compass image disappears. So when my user will drive exactly in North direction they know it by not seeing the image?! Very consistent!

I can center and rotate all by myself.

I'm a big boy, if I proved it with Google maps I can prove it everywhere?

Ok. I'm loosing that extra smooth animation for centering the map should I tell. But it just looks as it does for the Google maps, that would suffice. Seems to work just ok. So I was sitting and looking at that center and rotate I put together thinking "Stan, you are a genius, you proved it!".

As I tend to spend few minutes in this state, I felt how my lap is getting hotter, and hotter .... Oh oh, this is my beloved temperature based CPU profiling instrument! 

So here is the reason of why I'll be thinking twice now if I should use native Apple's map for the live map option, let me provide it here. On the top you'll see Apple's map consumption while centering and rotating and below it you'll see the same task executed by a Google map. If simulators doe this to my mac, I guess I can speculate the same CPU/battery consumption ratio will stay on real devices:


My story ends here, enough for this evening, I need to re-group, re-think, re-try, re-study ....








Sunday, September 29, 2013

iOS 7 vs iOS 6 memory consumption. OMG?

Known thing my app starts slower on iPhone 4 with iOS7 then it is on the same iPhone 4 with iOS6. This is pretty sad, but not the only sad thing...

As I'm profiling now the app before its soon to come release to AppStore, I got to the following screenshots for the fresh app start on iPhone 4 with iOS6:


Note the 1.55MB number. And now, the sad thing, same fresh start on iPhone5 with iOS7:


Noticed that 8.21MB?! I currently only have iOS7 installed on iPhone 5, so I can't compare iPhone 4 iOS7 vs iOS6 consumption, but I believe this iPhone 5/4 factor should not play any role here. It is just iOS thingy. 

While not that sad for iPhone 5/5s, it is pretty bad news for my users that are still on iPhone 4, apps are going to consume more. All apps, not only mine... App startup is becoming definitely heavier and those extra allocations in iOS7 are surely standing for more work done by iOS7 while starting the app.

Having read Steve's biography, it looks to me like a step away from the "saving lives" approach to plain MS's "throw more memory on it - problem solved".

May I be wrong? I hope!

Yours,
Stan.