Ben Poweski

A Simple Monte Carlo Experiment in R: More 𝜋

Monte Carlo simulations (or Monte Carlo methods) are a simple but useful technique to derive a solution to a problem by randomly guessing possible inputs. The definition is so broad that it’s likely you’ve used this technique without even realizing it.

The hello world problem for Monte Carlo applications is undoubtedly calculating approximations of 𝜋, albeit ineffeciently.

It’s Always About Darts

Before we can write software to do this we need to think about how to restate the question in probablistic terms. 𝜋 is simply the ratio of a circle’s circumference to its diamater.

We also know the area of circle is:

Suppose a circle was inscribed inside of a square.

We could then define the area of the square in terms of the circle’s diameter:

Or in terms of the circles radius:

Now, onto the darts. Imagine you’re throwing darts at the square. The universe is constrainted such that every dart lands randomly inside of the square.

The probability of a dart hitting the circle is the radio of the area of the circle over the square.

Which means:

Restated another way:

Now that we know a fourth of 𝜋, we can limit our trial to one quadrant because the ratio still holds true. This isn’t strictly nessecary but it simplies the program.

1
2
3
4
5
6
7
8
9
10
11
12
13
library("ggplot2")

set.seed(13)

make.pi <- function(n) {
    darts <- data.frame(x=runif(n), y=runif(n))
    darts$hit <- (darts$x^2 + darts$y^2) <= 1
    darts
}

calc.pi <- function(df) {
    4*sum(df$hit)/nrow(df)
}

Here we have the make.pi function that creates an R data.frame that holds the trials containing the X & Y coordinates of the dart position.

1
2
sim <- make.pi(10000)
ggplot(sim, aes(x, y, color=hit)) + geom_point() + ggtitle(calc.pi(sim))

This plot gives us a visualization of where each trial lands.

1
2
3
4
attempts <- seq(1, 100000, by=50)
estimated <- sapply(attempts, function(x) abs(calc.pi(make.pi(x)) - pi)/pi)

ggplot(data.frame(estimated=estimated, n=attempts)) + geom_line(aes(x=n, y=estimated)) + ylab("error %")

And finally we can check how well this solution scales with different size trials.

Using New Relic With Clojure

At getaroom.com we’ve used New Relic extensively for many years to monitor our Rails applications. We’ve recently began moving our backend infastructure to Clojure, maintaing the visibility New Relic gave us for our Rails environment was a must.

New Relic’s product is great, but the ability to add custom instrumention to a Clojure application is less than ideal. The primary method of custom instrumentation using New Relic’s Java agent is via custom annotations. Sean Corfield wrote an excellent post on how to get this to work using deftype.

Setup New Relic

Adding the New Relic API jar will permit custom metrics via the Java API. At getaroom we use it expose a number of metrics about our business of selling hotel rooms. These metrics can be displayed using custom dashboards along with many of the standard graphs available but they don’t provide the same insight that instrumentation of underlying function calls can provide.

Add New Relic API jar
1
2
:dependencies [...
               [newrelic.java-agent/newrelic-api "2.9.0"]]

Enabling New Relic requires launching the JVM using the -javaagent switch. The startup time of the agent leaves much to be desired. Given this, I’ve found it useful to conditionally enable New Relic via a leiningen profile.

Add New Relic Agent jar
1
2
{:profiles {:newrelic {:jvm-opts [ "-Dextensions.dir=<path>/extensions"
                                   "-javaagent:<path>/newrelic.jar"]}}}

Instrumentation of AOT Compiled Code

Adding a custom annotation in Clojure isn’t always practical. Let’s consider the case where we wanted to instrument the custom JSON library cheshire. If we were calling parse-string we could create functions that call through a method containing the custom Trace annotation as Sean Corfield describes here. This works but involves extra stubs used only to satisfy New Relic’s use of annotations.

If you’re willing to trade the downsides of AOT compilation for better instrumentation we can use New Relic’s custom XML extensions. We acheive this by using an AOT compiled uberjar. Currently, this trade-off is worth for us but YMMV.

Instrumenting Clojure Functions with Custom XML Extensions

In many cases the AOT compiled function has a name that is very easy to find. An anonymous function is not one of these cases.

The following is the roughly what the generated class looks like for the function cheshire.core/parse-string.

generated Cheshire signature
1
2
3
4
5
6
7
8
9
10
11
12
package cheshire;
...
public final class core$parse_string extends RestFn
{
  ...
  public Object doInvoke(Object string, Object p__4015) { ... }

  public int getRequiredArity()
  {
    return 1;
  }
}

Now that we know what the method signature looks like, we can add the custom extension XML.

Add the following XML to a file called cheshire.xml within the extensions folder of your New Relic install.

New Relic Extension XML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
<urn:extension xmlns:urn="newrelic-extension" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="newrelic-extension extension.xsd " name="Cheshire" version="1.0">
  <urn:instrumentation metricPrefix="Custom/Cheshire">
    <urn:pointcut>
      <urn:className>cheshire.core$parse_string</urn:className>
      <urn:method>
        <urn:name>doInvoke</urn:name>
        <urn:parameters>
          <urn:type>java.lang.Object</urn:type>
          <urn:type>java.lang.Object</urn:type>
        </urn:parameters>
      </urn:method>
    </urn:pointcut>
  </urn:instrumentation>
</urn:extension>

Once this is added and the application is restarted you should begin to see traces that include the generated class name for the cheshire.core/parse-string function. Additionally, using the excellent New Relic thread profiler you can easily find new functions that would benefit from visibility in New Relic traces.

Apple Push Notifications Toolkit for Ruby

Lately I’ve been working on an iPhone application that utilizes Push Notifications extensively. While Apple provides great documentation there is still a fair amount of work left up to the developer to send push notification in ordinance to their usage policy.

Since I’m using Rails for the backend I needed a way to maintain a single open SSL connection to prevent setup and tear down for every notification sent. To accomplish this I created an Event Machine based server deamon that that proxies all requests send by Rails application over a local, non SSL socket.

The proxy queues and sends each notification to the Apple notification servers. Additionally I’ve created a command line script to send notifications for testing purposes. My project Apn Server is hosted on Github.

You can add it to your Rails application via

1
2
config.gem "bpoweski-apnserver", :lib => 'apnserver', 
  :source => "http://gems.github.com"

You can use either a direct connection to Apple or one through the apnserverd proxy included in the gem. To configure directly to Apple’s server (and also setting up and tearing down a new connection each time). Add the following to your environment.rb.

1
2
3
ApnServer::Config.pem = '/path/to/pem'
  ApnServer::Config.host = 'gateway.push.apple.com'
  ApnServer::Config.port = 2195

To use the non SSL apnserverd proxy simply drop the PEM configuration option

1
2
ApnServer::Config.host = 'localhost'
  ApnServer::Config.port = 22195

To send a notification from Ruby

1
2
3
4
5
6
notification = ApnServer::Notification.new
  notification.device_token = Base64.decode64(apns_token)
  notification.alert = message
  notification.badge = 1
  notification.sound = 'default'
  notification.push

You can also send notifications via the line with

1
2
3
4
5
$ apnsend --server gateway.push.apple.com --port 2195 \
     --pem key.pem \
     --b64-token j92f12jh8lqcAwcOVeSIrsBxibaJ0xyCi8/AkmzNlk8= \
     --sound default \
     --alert Hello

Remove Bevel and Gloss Effect From iPhone Icon

After search for this a bit, it is a simple property inside of your Info.plist. To remove the bevel and gloss effect set the following property:

1
UIPrerenderedIcon true

For more properties check out the [Info.plist reference](http://developer.apple.com/iphone/library/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/ApplicationEnvironment/chapter_3_section_4.html#//apple_ref/doc/uid/TP40007072-CH7-SW15).

Parsing CSS With ANTLR

For a project I"m working on, we’ve used CSS syntax to describe styling on application objects. To accomplish this we created a parser using Antlr. Here is our implementation of the CSS core syntax. Unicode support is left out as it was not needed for our use case, but it should be pretty easy to add in.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
grammar CssCore;
options { language=Java; }

/*

Grammar taken from
http://www.w3.org/TR/REC-CSS2/syndata.html#tokenization

*/

//stylesheet  : [ CDO | CDC | S | statement ]*;
stylesheet
	:	(CDO|CDC|statement)*
	;


//statement   : ruleset | at-rule;
statement
	:	ruleset
	|	atRule
	;

//at-rule     : ATKEYWORD S* any* [ block | ';' S* ];
atRule	:	ATKEYWORD any* (block | SEMICOLON)
	;

//block       : '{' S* [ any | block | ATKEYWORD S* | ';' ]* '}' S*;
block	:	LBRACE (any|block|ATKEYWORD|SEMICOLON)* RBRACE 
	;

//ruleset     : selector? '{' S* declaration? [ ';' S* declaration? ]* '}' S*;

selector:	'*'
	|	'*'? (IDENT|'>'|'+'|CLASS|HASH)+
	;
	
	
ruleset	:	selector? LBRACE declaration? ( SEMICOLON declaration? )* RBRACE
	;

//selector    : any+;

//declaration : property ':' S* value;
declaration
	:	property COLON value
	;

//property    : IDENT S*;
property:	IDENT
	 ;

//value       : [ any | block | ATKEYWORD S* ]+;
value	:	(any|block|ATKEYWORD)*
          ;

//any         : [ IDENT | NUMBER | PERCENTAGE | DIMENSION | STRING
//              | DELIM | URI | HASH | UNICODE-RANGE | INCLUDES
//              | FUNCTION | DASHMATCH | '(' any* ')' | '[' any* ']' ] S*;
any	:	(	IDENT|NUMBER|PERCENTAGE|DIMENSION|STRING|
			HASH|INCLUDES|
			FUNCTION|DASHMATCH
			// TODO UNICODE_RANGE|DELIM|URI| '(' any* ')' | '[' any* ']' ] S*;
		)
	;


/* Tokens */

//IDENT 	{ident}
IDENT	:	F_IDENT
	;

//ATKEYWORD 	@{ident}
ATKEYWORD
	:	'@' F_IDENT
	;

//STRING 	{string}
STRING	:	F_STRING
	;

//HASH 	#{name}
HASH	:	'#' F_NAME
	;

//NUMBER 	{num}
NUMBER	:	F_NUM
	;

//PERCENTAGE 	{num}%
PERCENTAGE
	:	F_NUM '%'
	;

//DIMENSION 	{num}{ident}
DIMENSION
	:	F_NUM F_IDENT
	;

//URI 	url\({w}{string}{w}\)
//|url\({w}([!#$%&*-~]|{nonascii}|{escape})*{w}\)
//UNICODE-RANGE 	U\+[0-9A-F?]{1,6}(-[0-9A-F]{1,6})?


//CDO 	<!--
CDO	:	'<!--'
	;

//CDC 	-->
CDC	:	'-->'
	;


//; 	;
SEMICOLON
	:	';'
	;	
	
COLON	:	':'
	;	


//{ 	\{
LBRACE	:	'{'
	;

//} 	\}
RBRACE	:	'}'
	;

//( 	\(
LPAREN	:	'('
	;

//) 	\)
RPAREN	:	')'
	;

//[ 	\[
LBRACKET:	'['
	;

//] 	\]
RBRACKET:	']'
	;

//S 	[ \t\r\n\f]+
S	:	(' '|'\t'|'\r'|'\n'|'\f')+
		{ $channel=HIDDEN; }
	;

//COMMENT 	\/\*[^*]*\*+([^/][^*]*\*+)*\/
COMMENT	:	'/*' (options {greedy=false;} : .)*   '*/'
		{ $channel=HIDDEN; }
	;

//FUNCTION 	{ident}\(
FUNCTION:	F_IDENT '('
	;

//INCLUDES 	~=
INCLUDES:	'~='
	;

//DASHMATCH 	|=
DASHMATCH
	:	'|='
	;

//DELIM 	any other character not matched by the above rules

CLASS	:	'.' F_IDENT
	;


//ident 	{nmstart}{nmchar}*
fragment
F_IDENT	:	F_NMSTART F_NMCHAR*
	;

//name 	{nmchar}+
fragment
F_NAME	:	F_NMCHAR+
	;

//nmstart 	[a-zA-Z]|{nonascii}|{escape}
fragment
F_NMSTART
	:	(F_LETTER)
// TODO	add nonascii, escaped
	;

//nonascii	[^\0-\177]
//unicode 	\\[0-9a-f]{1,6}[ \n\r\t\f]?
//escape 	{unicode}|\\[ -~\200-\4177777]

//nmchar 	[a-z0-9-]|{nonascii}|{escape}
fragment
F_NMCHAR:	(F_LETTER|F_DIGIT|'-')
// TODO	add nonascii, escaped
	;

//num 	[0-9]+|[0-9]*\.[0-9]+
fragment
F_NUM	:	('0'..'9')+
	|	('0'..'9')* '.' ('0'..'9')+
	;


//string 	{string1}|{string2}
fragment
F_STRING:	F_STRING1
	|	F_STRING2
	;

//string1 	\"([\t !#$%&(-~]|\\{nl}|\'|{nonascii}|{escape})*\"
fragment
F_STRING1
	:	'"' ('\t'|' '|'!'|'#'|'$'|'%'|'&'|'\''|'.'|F_LETTER|F_DIGIT)* '"' 
	;
//string2 	\'([\t !#$%&(-~]|\\{nl}|\"|{nonascii}|{escape})*\'
fragment
F_STRING2
	:	'\'' ('\t'|' '|'!'|'#'|'$'|'%'|'&'|'.'|F_LETTER|F_DIGIT)* '\'' 
	;

//nl 	\n|\r\n|\r|\f
fragment
F_NL	:	'\n'
	|	'\r\n'
	|	'\r'
	|	'\f'
	;
	
fragment
F_LETTER:	'a'..'z'
	|	'A'..'Z'
	;
	
fragment
F_DIGIT	:	'0'..'9'
	;

//w 	[ \t\r\n\f]*
fragment
F_W	:	(' '|'\t'|'\r'|'\n'|'\f')*
	;

Securing Data Wtih OpenSSL and Ruby: Part One

For the most part Ruby has fantastic APIs. While there is an occasional wart here and there (I"m speaking to you [DateTime](http://www.ruby-forum.com/topic/146117)). In general, it doesn’t suck. The OpenSSL bindings for Ruby are no exception.

  1. Bob, meet Alice ##

To begin our cultural learnings of OpenSSL and Ruby, let’s take a look at source repository for the interpreter. In the [samples](http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/trunk/sample/openssl/) directory are some nice examples how the bindings work. It seems that some of the original code examples were never migrated from the RubyPKI project, but thats ok, you can still access [them](http://cvs.savannah.gnu.org/viewvc/rubypki/ossl2/examples/) here.

  1. The OpenSSL Digest Class ##

One of the most common things you’ll most likely need is to create a digest of a string of data. We can do this using by instantiating a Digest class then invoking the hexdigest method.

1
2
3
4
5
6
7
irb(main):001:0> require 'openssl'
=> true
irb(main):002:0> @sha1 = OpenSSL::Digest::SHA1.new("fooooo")
=> 58c00efa9bed725721b29f4b5f7864f0f191cad5
irb(main):003:0> @sha1.hexdigest
=> "58c00efa9bed725721b29f4b5f7864f0f191cad5"
irb(main):005:0>

Simple enough. But what is the deal with ‘digest’ versus ‘openssl/digest’? According to the ‘digest’ module is only used for backwards compatibility, so use the openssl version when possible.

  1. Available Digest Algorithms ##

According to the source, the Digest algorithms available are dependent upon the version of OpenSSL compiled with Ruby.

1
2
3
4
5
6
7
8
module OpenSSL
  class Digest

    alg = %w(DSS DSS1 MD2 MD4 MD5 MDC2 RIPEMD160 SHA SHA1)
    if OPENSSL_VERSION_NUMBER > 0x00908000
      alg += %w(SHA224 SHA256 SHA384 SHA512)
    end
    ...

Compiling Erlang Applications With Rake

The irony of Rake is that Ruby really doesn’t need it. This is not to say it isn’t useful to Ruby projects, quite the contrary. Where Rake shines is in building software for applications that require byte code and object code, such as Java or C centric projects.

One such language requiring byte code compilation is Erlang. The design of the runtime environment used by Erlang should be familiar to most Java or C# developers. Erlang uses a byte code compiler (erlc) and various application meta-data files (.app, .rel, .config). For my development work I use Erlide, the Eclipse based Erlang IDE. The standard project layout for an application created in Erlide is:

+ project
– ebin
– include
– src

When creating a Rake file for Erlang projects, I ran into a few problems with compiling the source into the separate ebin directory. Namely, dependencies where not being constructed as desired yielding a full rebuild every time I ran the build script! In the following example I dynamically create the file rules so that the bytecode (.beam) files will only be recompiled if the source (.erl) file is changed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
require 'rake'
require 'rake/clean'

CLEAN.include(['ebin/*.beam', '*.dump'])
SRC = FileList['src/**/*.erl']
BEAM = []

SRC.each do |fn|
  BEAM << dest = File.join('ebin', File.basename(fn).ext('beam'))
  file dest do
    sh "erlc -o ebin #{fn}" 
  end
end

namespace :erlang do
  desc "staring ermail" 
    task :run => [:compile] do
      sh("erl -noshell -pa ebin -s my_mod start")
    end

  desc "run tests" 
  task :test => BEAM do
    sh("erl -noshell -s test_my_mod test -s init stop")
  end
end

task :default => [:compile]
task :start => ['erlang:run']
task :compile => BEAM