Archive for the ‘Miscellaneous’ Category

A New Location Notice

May 15, 2018

For readers who usually try to find the latest article in this blog, this is to inform you that new posts will be placed on https://excellentcorner.com.

The old articles starting from this one below will still be here.

Thank you for your attention.

Advertisements

A Plea for Donation

May 3, 2018

First, I’d like to say thank you to all personnel involved in WordPress web application, so that I can post articles that can be shared to the world.

I’ve started to create the article into this platform since 2008 and now it is already have more than 100 articles and still counting 🙂

This website is about problem and solution in the field of IT (Information Technology) ranging from Windows, Android to any other open sources platforms.

You can see that in the span of the year 2008 until now, the average article each month is about one or two articles.

It is because of the nature of the subject, which is about problem and its solution, sometimes, the solution will came about a month after the problem is clearly identified.

One example is my research on Android’s SIGILL problem in Android Native Debugging. Much of the time is to seek out information and provide necessary conditions to tackle the problem at hand.

The topic of SIGILL itself is requires about 5 articles starting from problem statement to its solution.

So, it takes time to create article with this kind of situation.

If you find this article is helpful, inspiring, saves your time and money, you can consider to provide donation to my Paypal account at surya_rakanta[at]yahoo.com.

But this doesn’t mean that you obliged to provide donation when reading at my website, it’s optional.

How to Check Memory Leak in ASP.NET Application

February 22, 2018

The indication of excessive usage of memory in ASP.NET application can be seen in the anomaly of memory size in w3wp.exe on Windows Server task manager.

Based on experiences, the unusual memory size is caused by using SELECT * criteria without any limiting parameter, and the retrieve records is in range of hundred thousand of records.

But the question is, which SELECT * statement ?

The first step is performing memory dump of the w3wp.exe in question by using WinDBG as follows:

In this case, the resulting file size is whooping 900 MB.

This file then is moved to the local computer for further analysis, and then this is opened using WinDBG. Activate SOS.DLL for appropriate .NET version as follows:

Then use the very popular command called dumpheap as follows:

Here it is shown that there is DataRow object that contains an unusually large rows of 294656. Surely this is very big for a normal and optimized application.

Let’s see more detail here:

WinDBG will show tons of object data, so you can use CTRL-BREAK and check just one item only because, each objects are the same.

Just take one of them to be examined:

One of interesting object to be examined is the column object:

Check the list object:

Then check the list item:

Take one to be examined:

Check the column name:

Based on the column name we can inferred the table name involved, but which select statement that retrieve the unusual amount of records ?

Let’s check again the result of dumpheap for SqlDataAdapter as follows:

There are 7 objects to be examined, this should be checked one by one:

The first one clearly do not cause memory leak, so the above step should be repeated for subsequent objects. Until it is found as follows:

The above query clearly will cause memory leak, especially when the data rows already reaching thousands of records. For the above case, it will gives about 300000 records and will eating up much of Windows OS memory in W3WP.EXE.

To check what kind of program that runs the above query, you can examine again the result of object dump as follows:

Perform the revision of the program’s routine and memory leak is now solved !

RBMware (*.RBM) File Format

January 17, 2018

RBMware is a data analysis software. And the RBM itself is Reliability Based Maintenance. But I don’t want to explain more regarding how to use this software.

This article will focus on how to access the database used in RBMware application, especially the data used by OilAnalysis.exe program.

The database is using *.RBM file extension and it is maintained by one of the sofware component called DAF42.DLL.

In user perspective, the RBM data contains hierarchical format as follows:

Here you can see that there are 5 (five) level of hierarchical data: Database, Area, Equipment, Point and the sample data.

Using low level perspective, the database consists of blocks of 512 bytes size called a Record, and it is usually access using DAF42.DLL’s ReadRecord by assigning the record number.

To access each block based on record number, you can use the (n-1) * 512 formula. For example, when the record number is 0x46 or 70, then (70 – 1) * 512 we get 35328 or 0x8A00 which is the block or physical sector number of the record being accessed.

The database can contain zero or many areas, but how to access the record that contains area information ? This is done by accessing the data at sector 0 which contains data as follows:

So the physical sector for the area records is located at record number 0x46, and when translated to physical sector using the above formula, we get 0x8A00:

You can see that it contains the fix sized long description and short description. Each block can contain maximum of 22 items of long description and short description. So what if there’s more than 22 records of area ? It is maintained using the dword data (green one) that will contain record number of the next area data to be retrieved.

Now each area can have zero or many equipment records. This is done by retrieving dword data marked with yellow color (see above). And it is a collection of intermediate record called stdg (or read as gdts at low level) for each corresponding area.

So, for example when the corresponding block has 3 areas, then the dword collection will has 3 items, each points to associated stdg for the related area.

Let’s traverse to first stdg record for first area which is the record number 0xB6 (182) which is (182 – 1) * 512 = 92672 or 0x16A00:

The dword data mark with red color denotes record number for the equipment block. Let’s perform some calculation on 0xC4 (196) which is (196 – 1) * 512 = 99840 or 0x18600:

It has the same characteristics as area record, so the dword denoted with blue one will point to next equipment record when there are more than certain limits.

To retrieve the point record related to each equipment, you can refer to collection of dwords denoted with yellow color called mcdg or gdcm at low level view. Let’s perform some calculation on 0xC6 (198) which is (198 – 1) * 512 = 100864 or 0x18A00:

The mcdg will contain record number to mpig or gipm (denoted with blue color). 0xE4 (228) which is (228 – 1) * 512 = 116224 or 0x1C600:

The mpig record contains collection of point description record (mpdo or opdm denoted by yellow color) and also point short description (code). For example at record number 0xE8 (232) which is (232 – 1) * 512 = 118272 or 0x1CE00 we have:

Each point has record number (blue color above) that points to dcod or odcd record that contains start (red color below) and end (blue color below) record number for point samples data. For 0xEB (235) which is (235 – 1) * 512 = 119808 or 0x1D400 we have:

From the above structure, the start record number for sample point is located at 0x121 which is (289 – 1) * 512 = 147456 or 0x24000 we have:

From the above point sample data or tddo (oddt) we can retrieve record number for next sample for corresponding point (green color), sample data time stamp (blue color), the sample description (red color) and array of dwords 32 bit float data in the IEEE-754 format that will show up on user perspective as follows:

The dword arrays will corresponds to Analyzing Parameter (AP) that can be customized by the user. For the above example, the AP Set is Aluminum, Antimony, Barium, etc. How each of the dword arrays corresponds to the AP Set ?

Let’s take sample 219 for example:

The floating point 0x40C00000 is floating point for 6 for AP Aluminum, the next one will corresponds to Barium, etc. This correlation can be retrieved using record called apdo or odpa.

The record number for apdo is retrieved from apig or gipa record which is a constant of 12 (0xC). So by passing 0xC to ReadRecord we arrive at apig record as follows:

From the above apig record, the apdo record number is denoted with red color so we have 0xA9 which is (169 – 1) * 512 = 86016 or 0x15000:

Based on these parameter strings, then I can provide corresponding relation between array of values from apdo record to the parameter description.

Based on the above information, it is possible to create an application to provide tree traversing and data access for some necessary purposes.

How to Get Property Name in Compiled Visual Basic Applications

December 19, 2017

Suppose I created a small program in Visual Basic form that contains one button, whose click event is coded as follows:

Then the compiled Visual Basic program for the form’s name assignment above will look like this:

Suppose then I revised the above coding to:

Then the generated program will revised to:

Clearly there’s some mapping between the call offset that is generated and the property name in the source program. So, if I can somehow perform the reverse operation between the call offset to the property name, then I can apply it to the unknown compiled program to determine what kind of the property that the program is accessed.

After performing some detailed analysis, it is revealed that the above offset has some relation to the Dispatch ID and it is processed using VB6.EXE’s unknown interface method.

When I tried to view the interface methods of VB6.EXE using OLEVIEW.exe it is failed to do so, but there’s file called VB6.OLB that can be successfully viewed by OLEVIEW.exe program.

And here is the part of the content that deals with form’s method:

What’s interesting is that it is missing the Dispatch ID that usually exist in regular application’s type library functions. As you can see from the above, the Name property is the first one declared in the method interface, and 0x48 is actually the lowest magic number value.

The next method name can be deduced by adding 0x8 offset to the starting number, so for example above, the value 0x50 will corresponds to Caption property etc.

Using the above fact, then I can create the mapping table between the property and it is calling offset, so that it can be used to deduce the property name that is used from certain unknown applications that is developed using Visual Basic compiler.

How to Install NopCommerce 3.90

November 15, 2017

Since the downloaded 3.90 source from:

https://github.com/nopSolutions/nopCommerce/releases/

Can be compiled using Visual Studio 2013 without any issues, this article will focused on deployment and installation of nopCommerce.

The target server machine is using Windows Server 2012 R2 64 bit and IIS 8.0.

First, create the virtual directory called NoC:

And assigned the proper Application Pool to it:

Then copy the entire compiled version of nopCommerce in D:\Projects\Mvc\NoC\3.90\src\Presentation\Nop.Web to the destination server folder.

Then fire up the browser using this link:

That’s all folks 🙂

How to Compile and Run Open Whisper Systems’ Signal Server (TextSecure Server) (Part 2)

August 9, 2017

Last time, when I try to activate Signal Server, I got this:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
null

So, it exited the loop, and only giving a “null” as an error message. This is the most cryptic error I’ve ever get in debugging experieces.

Checking the exact source of exception reveals it is java.lang.NullPointerException.

So, the call stack can be easily obtained by breaking into NullPointerException in the java sources:

And I have:

Let’s perform of analysis of this class:

APNSender apnSender = new APNSender(accountsManager, config.getApnConfiguration());

private static PrivateKey initializePrivateKey(String pemKey)

Let’s see the content of pemKey:

lkajsdf

The content of pemKey appears to be obtained from:

Since I just create the same dummy variables for the above configuration variables, let’s differentiate it, and I have that pemKey is initialized using pushKey configuration item:

PEMReader reader = new PEMReader(new InputStreamReader(new ByteArrayInputStream(pemKey.getBytes())));

I assumed that to get the pushKey from Apple required some dollar involvement, so, let’s just create a dummy PEM Key just to make this class happy.

import org.bouncycastle.openssl.PEMReader

So, this is part of the utility created by bouncycastle, let’s get some information regarding this first. OK, so this is a PEM Certificate, let’s try to generate it:

C:\OpenSSL-Win32\bin>openssl genrsa -out rsa_1024_priv.pem 1024
Generating RSA private key, 1024 bit long modulus
………++++++
…..++++++
e is 65537 (0x10001)

Run the server, and I have:

D:\Projects\AndroidProgram\Signal\Server>java -Xdebug -Xrunjdwp:transport=dt_soc
ket,server=y,address=8880,suspend=y -jar target\TextSecureServer-1.65.jar serve
r config/textsecure.yml
Listening for transport dt_socket at address: 8880
—–END RSA PRIVATE KEY not found

No matter what I try to fiddle with the key, I got this same error message. I believe the format I’ve passed to the configuration is OK. But let’s see whether it is generated some exception again.

This time, I got java.io.IOException. Repeat the above procedure, I have:

But when I try to trap the IOException, there’s an interesting IOException just BEFORE this exception. So, let’s check it out first:

\C:\Program Files (x86)\Java\jdk1.8.0_131\jre\lib\sunrsasign.jar

Seems, it has some relation with RSA signing. From the findjar website, I’ve found that is already in rt.jar so, I think I’ll stop pursue this issue for the moment.

Let’s come back to the above rather persistent error message to understand what it try to process from the given private key.

Retrieve bouncycastle module sources from:

http://central.maven.org/maven2/org/bouncycastle/bcprov-jdk16/1.46

Which is the dependency currently used by my TextSecureServer java program at this moment.

But NetBeans complains as follows:

Not able to submit breakpoint LineBreakpoint PemReader.java : 54, reason: Line number information is missing in the class file org.bouncycastle.util.io.pem.PemReader.

Let’s verify:

This is the one that has line number information:

D:\temp\dw\io\dropwizard>javap -l application.class
Compiled from “Application.java”
public abstract class io.dropwizard.Application {
protected io.dropwizard.Application();
LineNumberTable:
line 25: 0
line 27: 4
line 28: 11

And indeed, the given class in the jar does not have line number information:

D:\temp\bc\org\bouncycastle\openssl>javap -l pemreader.class
public class org.bouncycastle.openssl.PEMReader extends org.bouncycastle.util.io
.pem.PemReader {
public org.bouncycastle.openssl.PEMReader(java.io.Reader);

public org.bouncycastle.openssl.PEMReader(java.io.Reader, org.bouncycastle.ope
nssl.PasswordFinder);

Try to find compatible environment for BouncyCastle’s version 1.46 compilation proves to be difficult. After perform some research, I decided to compile the version 1.57 instead.

D:\Projects\JavaProgram\BouncyCastle>gradlew build -x lint
Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain

This can be resolved by copying existing gradle folder in the usable projects into this project folder.

Now I have:

D:\Projects\JavaProgram\BouncyCastle>gradlew build -x lint
Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be reus
ed, use –status for details
0% CONFIGURING [42s]
> root project > Resolve dependencies :classpath > gradle-cobertura-plugin-2.2.

It failed to download the above package. So again, using proxy will alleviate the problem.

But the result of version 1.57 is somewhat vague, which has many JARs that is not compatible with the intended version.

So, I have to resort to create a customized gradle build system from older 1.46 sources in D:\Projects\JavaProgram\BouncyCastle2, and after a successful compile, I verify it that it is indeed it has line number information in it:

D:\temp>javap -l pemreader.class
Compiled from “PemReader.java”
public class org.bouncycastle.util.io.pem.PemReader extends java.io.BufferedRead
er {
public org.bouncycastle.util.io.pem.PemReader(java.io.Reader);
LineNumberTable:
line 19: 0
line 20: 5
LocalVariableTable:
Start Length Slot Name Signature
0 6 0 this Lorg/bouncycastle/util/io/pem/PemReader;
0 6 1 reader Ljava/io/Reader;

So, now I try to replace the 1.46 version in my local maven repository and perform the recompiling.

But to my surprises, I found many errors.

This is strange, because the first time I try to compile it is a success. Why the second compilation process is not ?

First, I suspect maybe it is the cause of my replacing of BouncyCastle 1.46, so I revert back to the old jar and try to re-compile with exact same error message:


[ERROR] /D:/Projects/AndroidProgram/Signal/Server/src/main/java/java/util/function/LongUnaryOperator.java:[64,5] default methods are not supported in -source 1.7
(use -source 8 or higher to enable default methods)
[ERROR] /D:/Projects/AndroidProgram/Signal/Server/src/main/java/java/util/function/LongUnaryOperator.java:[66,25] lambda expressions are not supported in -source 1.7
(use -source 8 or higher to enable lambda expressions)
[ERROR] /D:/Projects/AndroidProgram/Signal/Server/src/main/java/java/util/function/LongUnaryOperator.java:[92,38] static interface methods are not supported in -source 1.7
(use -source 8 or higher to enable static interface methods)

Try to change the version as suggested by the error message, i.e change to 1.8 proves to be generated more cryptic errors. So I revert back to the old configuration, and I decided to compare the original sources to the one I try to re-compile, and I found mysterious java folder inside the sources I try to re-compile.

And suddenly I realized it is my mistake this time, because I accidentally use this source folder as a NetBeans debugging, and added unnecessary sources into the folder.

Next time I should place the debugging source files into another folder. After removing the offending folders, and try to recompile:

[INFO] ———————
[INFO] BUILD SUCCESS
[INFO] ———————

It is OK with the version 1.46 of BouncyCastle utility. So, let’s replace it now with one with line number information and see what happens when I try to re-compile, again it is a success.

To create a separate source, I created a new folder at D:\Projects\NetBeans\SignalServer to host the new location of NetBeans project.

After some studying of NetBeans to try to separate the sources for debugging and the actual program so that the above compiling error can be avoided, I realized, it is very difficult to achieve that condition.

Later on, I found that using “Remote Attach” to add new Source Root will solve the problem:

The Java and any other JARs that required source code debugging then can be placed into D:\Projects\NetBeans\Debug\src. Now, let’s start to analyze the BouncyCastle error.

First, I try to perform breakpoint at the PemReader’s loadObject, and here is the result:

The readLine function suddenly returning with null, so it end up with the —–END RSA PRIVATE KEY not found error. This indicated that the buffer is not received any of the data. Hmm, fishy.

After checking the readPemObject function of PemReader.java, I realized that the passed string should contain Carriage Return and Line Feed. But how to encode it into YAML object ?

Try to pass \r and \n proves to be futile, because readLine inside this function returns one line:

After some study of YAML using this format will solve the issue:

Now I have:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65.
jar server config/textsecure.yml
org/eclipse/jetty/alpn/ALPN$Provider

The cause of the above exit is java.lang.NoClassDefFoundError. Try to perform break point, I have:

at java.lang.NoClassDefFoundError.(NoClassDefFoundError.java:59)
at io.netty.handler.ssl.JdkAlpnApplicationProtocolNegotiator$1.(JdkAlpnApplicationProtocolNegotiator.java:26)
at io.netty.handler.ssl.JdkAlpnApplicationProtocolNegotiator.(JdkAlpnApplicationProtocolNegotiator.java:24)
at io.netty.handler.ssl.JdkSslContext.toNegotiator(JdkSslContext.java:285)
at io.netty.handler.ssl.JdkSslClientContext.(JdkSslClientContext.java:261)
at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:751)
at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:418)
at com.relayrides.pushy.apns.ApnsClientBuilder.build(ApnsClientBuilder.java:408)
at org.whispersystems.textsecuregcm.push.RetryingApnsClient.(RetryingApnsClient.java:63)
at org.whispersystems.textsecuregcm.push.APNSender.(APNSender.java:61)
at org.whispersystems.textsecuregcm.WhisperServerService.run(WhisperServerService.java:176)
at org.whispersystems.textsecuregcm.WhisperServerService.run(WhisperServerService.java:111)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:85)
at io.dropwizard.cli.Cli.run(Cli.java:74)
at io.dropwizard.Application.run(Application.java:89)
at org.whispersystems.textsecuregcm.WhisperServerService.main(WhisperServerService.java:276)

The above exception is tripped by the statement:

if (!JdkAlpnSslEngine.isAvailable()) in JdkAlpnApplicationProtocolNegotiator.java, which is caused by un-initialized JdkAlpnSslEngine

But why it has connection to the above org/eclipse/jetty/alpn/ALPN$Provider ? Checking the source code of JdkAlpnSslEngine.java, it is indeed there’s reference to this class:

import org.eclipse.jetty.alpn.ALPN

Performing checking to the binary JARs also confirm the non-existent of this class.

Some statement from the web regarding this issue in https://stackoverflow.com/questions/39856972/http-2-java-8-jetty-and-alpn:

Jetty’s ALPN boot jar works with both OpenJDK and Oracle’s JDK (which is based on OpenJDK).

Jetty’s ALPN boot jar must be in the boot classpath, not the regular classpath, like the documentation you linked says.

As such, you must not declare it as a dependency in your pom.xml files (there is no need to, like there is no need for you to specify a dependency on the JDK classes).

JDK 9 will have ALPN support native, there is already some work in that direction.

For java version java version “1.8.0_131” requires 8.1.11.v20170118.

Configuring command line to:

java -Xbootclasspath/p:target\alpn-boot-8.1.11.v20170118.jar -jar target\TextSecureServer-1.65.jar server config/textsecure.yml

D:\Projects\AndroidProgram\Signal\Server>java -Xbootclasspath/p:target\alpn-boot
-8.1.11.v20170118.jar -jar target\TextSecureServer-1.65.jar server config/textse
cure.yml

Will solve this issue, but now the server is continuous loop. But seems there’s no information, logs or anything to indicate what is it that is doing. Using ProcMon to find out interesting files created reveals:

D:\tmp\textsecureshserver.log

Inside this file is a wealth of information regarding the server’s activities. Hmm, good job Open Whisper Systems. Although there’s some issue should be fixed, for example a more friendly error message and some initial information of where to search vital information such as log, etc.

But I think this is the way how open source community works.

After a while, the server stops by itself:

D:\Projects\AndroidProgram\Signal\Server>java -Xbootclasspath/p:target\alpn-boot
-8.1.11.v20170118.jar -jar target\TextSecureServer-1.65.jar server config/textse
cure.yml
Multiple exceptions

Hmm, interesting, but I will stop for the moment.

How to Compile and Run Open Whisper Systems’ Signal Server (TextSecure Server)

August 7, 2017

The readme.md content from https://github.com/WhisperSystems/Signal-Server do not give a slightest hint of how to build the source.

Reading some hints in http://debabhishek.com/writes/Installing-and-Running-TextSecure-Signal-Server-on-Windows/. It turns out it should be compiled using Maven.

So, let’s download maven first. Then perform some sanity check:

C:\Windows\System32>mvn -version
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T02:39:0
6+07:00)
Maven home: C:\Maven\bin\..
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: C:\PROGRA~2\Java\jdk1.8.0_131\jre
Default locale: en_US, platform encoding: Cp1252
OS name: “windows 7”, version: “6.1”, arch: “x86”, family: “windows”

Now, let’s compile:

D:\Projects\AndroidProgram\Signal\Server>mvn clean install -DskipTests
[INFO] Scanning for projects…
[WARNING] The project org.whispersystems.textsecure:TextSecureServer:jar:1.65 us
es prerequisites which is only intended for maven-plugin projects but not for no
n maven-plugin projects. For such purposes you should use the maven-enforcer-plu
gin. See https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.ht
ml
[INFO]
[INFO] ————————————————————————
[INFO] Building TextSecureServer 1.65
[INFO] ————————————————————————
Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven
-source-plugin/2.2.1/maven-source-plugin-2.2.1.pom
[INFO] ————————————————————————
[INFO] BUILD FAILURE
[INFO] ————————————————————————
[INFO] Total time: 22.536 s
[INFO] Finished at: 2017-08-03T09:51:24+07:00
[INFO] Final Memory: 7M/17M
[INFO] ————————————————————————
[ERROR] Plugin org.apache.maven.plugins:maven-source-plugin:2.2.1 or one of its
dependencies could not be resolved: Failed to read artifact descriptor for org.a
pache.maven.plugins:maven-source-plugin:jar:2.2.1: Could not transfer artifact o
rg.apache.maven.plugins:maven-source-plugin:pom:2.2.1 from/to central (https://r
epo.maven.apache.org/maven2): Connect to repo.maven.apache.org:443 [repo.maven.a
pache.org/151.101.40.215] failed: Connection timed out: connect -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResoluti
onException

D:\Projects\AndroidProgram\Signal\Server>

So, perform changes in the Maven’s settings.xml:

After some lengthy POM and JARs download, I have:

[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 24:57 min
[INFO] Finished at: 2017-08-03T11:06:09+07:00
[INFO] Final Memory: 30M/104M
[INFO] ————————————————————————

Let’s run it:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65.jar

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65.
jar
usage: java -jar TextSecureServer-1.65.jar
[-h] [-v]
{server,check,directory,vacuum,trim,stats,rmuser,accountdb,messagedb}

positional arguments:
{server,check,directory,vacuum,trim,stats,rmuser,accountdb,messagedb}
available commands

optional arguments:
-h, –help show this help message and exit
-v, –version show the application version and exit

Let’s run it using the configuration given from the above website:

java -jar target\TextSecureServer-1.65.jar server config/textsecure.yml

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has an error:
* Malformed YAML at line: 8, column: 3; while scanning a simple key
in ‘reader’, line 8, column 1:
+33756796138 #fake
^
could not find expected ‘:’
in ‘reader’, line 9, column 1:
localDomain: foo.org
^

at [Source: java.io.FileInputStream@9175d8; line: 7, column: 2]

So, let’s debug it:

java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=8880,suspend=y -jar target\TextSecureServer-1.65.jar server config/textsecure.yml

The command line result:

D:\Projects\AndroidProgram\Signal\Server>java -Xdebug -Xrunjdwp:transport=dt_soc
ket,server=y,address=8880,suspend=y -jar target\TextSecureServer-1.65.jar serve
r config/textsecure.yml
Listening for transport dt_socket at address: 8888
config/textsecure.yml has an error:
* Malformed YAML at line: 8, column: 3; while scanning a simple key
in ‘reader’, line 8, column 1:
+33756796138 #fake
^
could not find expected ‘:’
in ‘reader’, line 9, column 1:
localDomain: foo.org
^
at [Source: java.io.FileInputStream@f76a6; line: 7, column: 2]

There’s no more helpful information. So, let’s find at what routine or module that give the above error, i.e. the Malformed YAML error string. This is done by extracting files in the JAR and perform binary search into the files by using utilities, such as WinHex. It takes some time, but I think this is one of possible starting point.

But the search turns out … nothing, either by using ASCII or UNICODE format. Hmm, so what to do ?

Try to do googling regarding the error message, I realized that the error location could be somewhere at YamlConfigurationFactory class of io.dropwizard java class utility.

Let’s obtain this source, but first I have to consult the maven repository of the version being used in this java application, and it turns out using v1.1.0.

Try to obtain the source from http://central.maven.org/maven2/io/dropwizard/dropwizard-configuration/1.1.0/dropwizard-configuration-1.1.0-sources.jar proves to be diffcult because what I got is a connection reset error.

After the source code is obtained, and placed it in proper folder on the Netbeans project, I place the breakpoint to this routine:

It is a hit, but the Netbeans is hang when I try to view the call stack.

What I only got is some information like this:

But this should be enough. Upon restarting the Netbeans, turns out to be ok.

After importing proper source code and perform some step traces, I have:

Where the location of the error is at BaseConfigurationFactory.java in io.dropwizard class:

The reason that I do not find “Malformed YAML” error message is because error string is generated programmatically, where as when I try to search for string Malformed I find many instances in the binary files.

So, let’s examine more detail of this error, which is actually originated from MarkedYAMLException:

Checking further:

This is because the config I copied from given by above website is indeed malformed, the spaces is missing, and after fixing the TextSecure.yml config file:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has an error:
* Unrecognized field at: push.host
Did you mean?:
– queueSize

No matter what I modify in the file, I always get this constant message error, and there’s a mysterious suggestion of queueSize. So let’s fire our binary search again shall we ?

Binary search is suggested at BaseConfigurationFactory.java and the suggestion is retrieved using e.getKnownPropertyIds().stream().

OK now, let’s enter this property using blank value and remove the unknown property, and we have:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has an error:
* Unrecognized field at: websocket
Did you mean?:
– webSocket
– cache
– redphone
– server
– metrics
[16 more]

To retrieve complete properties, I have to resort to debugger which is:

s3,cache,federation,read_database,webSocket,testDevices, twilio, directory, gcm, httpClient, push, database, redphone, apn, metrics, messageStore, turn, logging, server

The above keyword should be the complete keyword for the root property configuration options. There’s typo in the given sample text secure configuration from the above website.

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has an error:
* Unrecognized field at: webSocket.enabled
Did you mean?:
– requestLog

At this point, I started to wonder the validity of the given configuration from the above website.

After browsing some of the codings, then I realized, that the purpose of the parser is to try to map the text in to the class, at the root class is named WhisperServerConfiguration.java.

After more fiddling with the TextSecure.yml, I finally get:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has the following errors:
* apn may not be null
* gcm may not be null
* turn may not be null
* webSocket.requestLog may not be null

Let’s first tackle the webSocket configuration issue by passing requestLog: abc, since I don’t know the purpose that option yet, and I have:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has an error:
* Failed to parse configuration at: webSocket.requestLog; Can not construct in
stance of io.dropwizard.request.logging.LogbackAccessRequestLogFactory: no Strin
g-argument constructor/factory method to deserialize from String value (‘abc’)
at [Source: N/A; line: -1, column: -1] (through reference chain: org.whispersys
tems.textsecuregcm.WhisperServerConfiguration[“webSocket”]->org.whispersystems.w
ebsocket.configuration.WebSocketConfiguration[“requestLog”])

After some head scratching and source code review, I realized, this must be configuration item from io.dropwizard utility and I found that using this:

Will stop the complain. But what if I purposely mistyped the type ? In this case, I have:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has an error:
* Failed to parse configuration at: webSocket.requestLog.appenders; Could not
resolve type id ‘consolex’ into a subtype of [simple type, class io.dropwizard.l
ogging.AppenderFactory]: known type ids
= [AppenderFactory, console, file, papertrail, syslog]
at [Source: N/A; line: -1, column: -1] (through reference chain: org.whispersys
tems.textsecuregcm.WhisperServerConfiguration[“webSocket”]->org.whispersystems.w
ebsocket.configuration.WebSocketConfiguration[“requestLog”]->io.dropwizard.reque
st.logging.LogbackAccessRequestLogFactory[“appenders”])

Hmm, interesting. There are indeed other types of requestLog option, but let’s revert it back to console, now I have:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
config/textsecure.yml has the following errors:
* apn may not be null
* gcm may not be null
* turn may not be null

After proper settings of dummy TextSecure.yml file I have:

D:\Projects\AndroidProgram\Signal\Server>java -jar target\TextSecureServer-1.65
.jar server config/textsecure.yml
null

No exceptions, anything whatsoever, just “null”. Well, I will stop here for a moment.

Bootstrap Dialog Table Look Up Example

May 10, 2017

Here is the table look up example in action. When the page is properly setup it will show one button as follows:

When the button is clicked it will show the look up dialog box:

The data is retrieved from the server using AJAX based on user action, such as a click on table header, etc. The table is actually the extension of the original jQuery.DataTable.js open source program.

The table can be dynamically built using SELECTOR keyword:

The programmer only responsible for performing the codes inside the server side ASPX handler for given selector keyword, such as table header definition, paging request and display data. This is included in the sample NewInquiry.aspx page.

User can hover to the table and perform selection by clicking on to the table, and it will show the alert box:

This sample program is created using ASP.NET, so if you are interested to see how this page in action, there are several pre-requisite as follows:

1. The data source is using SQL Server 2005
2. Windows Server 2003 or higher with Framework 4

Sample data is included in the “data” folder on the source package.

Source files and supporting executable as follows:

https://github.com/surya-rakanta/BootstrapLookUp.git

How to Decompile MultiCharts SEF File

March 21, 2017

This article will explore whether it is possible to perform decompilation of MultiCharts’ *.SEF file back to its original Power Language Script, a variant of Easy Language Script. This is one of trading applications just like TradeStation.

The *.SEF file is created by using the “Export Read Only” menu in MultiCharts Power Language Editor (PLEdit.exe). At first glance, the content of generated file consist some header information and the actual executable file.

Let’s find whether the editor is keeping the Intermediate code, or hopefully an encrypted source code just like TradeStation’s *.ELD file.

This is done by creating an increasing more complex script and see the changes inside the *.SEF file. So, after several steps of creating, compiling and examining the file content, I realized that the PLEdit program is in fact, do not try to preserve the intermediate code, let alone the encrypted source code.

The only information is the actual executable file that is located just after the header portion of the file which typically like this:

There’s a MZ signature that denotes that portion of the block to the end of *.SEF file is actually the executable file. Using the Hex Editor I can move out the block into separate file and try to dissasemble it:

This executable file is actually a form of a DLL file because it has some exported functions, one of which is called “Create”. When I try to execute the compiled script in MultiCharts program by using the “Insert Study …” menu, it will try to load the generated i_myTesting.dll.

The myTesting is the is the name that I declared when creating the script of type Indicator (hence the i_ prefix). This is evident by viewing the Load DLL output from WinDBG program:

When “Insert Study…” is executed for myTesting script, the program will call the DLL’s Create exported function.

So, I landed on a rather hopeless case of machine code that only the computer’s CPU and MultiCharts program can execute faithfully. At this point, I try to googling out for this issue. I found that certain website does actually stated that they can decompile manually from the machine code to the script source code.

I try to communicate with one of the support of this service, and get a positive response, that indeed, yes, they can decompile and it require about 3 or 4 weeks and some compensation.

How can they do that ? This let me pause and think, maybe I do not perform a more in depth analysis. Then I have another activities to perform, but the question remains in my head. It’s like they can provide something from nothing.

So, I decided to take a more close look, by try to identify how the PLEditor generate it’s DLL file from the script. I created the most simple Indicator script called myTesting and it’s content as follows:

Then I realized that the PLEditor.exe, actually compile it’s file by using the process called StudyServer.exe.

By examining the modules loaded by StudyServer.exe, I’ve found the curious file called GccWrapServer.dll. By examining more closely, it is revealed that it is actually a COM object and it has a methods called Compile and SetCompilerSettings method. This is evident by viewing its type library using OLE View program:

From the above picture, I can see that the Compile method accepts the CPP file name and generated DLL file name. By setting the breakpoint into the Compile method, what I get from these two parameter as follows:

From the above execution code, I can see that the ecx register contains the CPP file name:

Folder content as follows:

After the Compile finishes, the *.cpp file is removed, so I try to obtained it while the program is suspended by the debugger and the content of CPP file as follows:

There are many information revealed by this generated CPP and its corresponding *.o file, one is the parameter information. So the above Create export function calls can be identified as follows:

The above rather hopeless code starts to have some meaning, while it is just a start, then the decompilation of MultiChart’s *.SEF file suddenly become possible. Much works still to be done, it’s hard, but it is indeed possible 🙂