Sunday, October 27, 2013

What I think is wrong with Kickstarter

With the announcement from Double Fine that they're almost out of their kickstarter money and not even half through the game, people have started to question kickstarter and the idea of crowdfunding. Adam Sessler has some great words to say, so does Greg Miller's roommate Colin.

I did support the Double Fine adventure game, aka Broken Age. I am a sucker for good ol' Point-n-Click games and though the idea sounded awesome since 'Day of the Tentacle' is one of my favorite games. But when Double Fine started a new kickstarter for Massive Chalice, I was very sceptical thinking, 'shouldn't you prove that your first kickstarter is successful before asking for more money to a new project?'. And that's the thing, most of these projects from established developers plays on the fact that peoples are nostalgic for a product. Seeing Brian Fargo trying to sell his idea of Wasteland 2 to a kid in the role as a publisher is pretty funny, and the reaction from the kid when Brian mentions games he worked on and the kid has never heard of any of them, but isn't that the thing? Big part of the people that play games today haven't heard of these games! (I didn't know about Wasteland). With that, I'm not saying that old franchises should be buried and forgotten. But they shouldn't be update to todays games by just turning them into FPSes. I think, like Adam Sessler says, Kickstarter could work as a way for helping unkown people create vertical slices and prototypes for trying to sell their idea the regular way. Even if I do believe people that haven't contributed to the Broken Age kickstarter will buy the game.

I have some ideas that I think should be overlooked on the kickstarter model. 

Don't allow overfunding:
Its great to have ambitions and goals, but if you ask for X amount of dollars from an investor, they won't throw more money at you, they expect that you've made a budget to reach the goal. Some projects do show their overfunding budget, but I think kickstarter should allow these by the same idea that original goals work, If you have a overfunding goal at $750,000 after your original $500k, you would only get it when you reach that goal. 

I think it should work as follows:

  • When you back a project, you say "I'm ready to back this project with $40". Until the project is fully backed, this is the amount of money you will back the project with.
  • When the project is fully backed, any more backers willing to chip in dollars will lower your backing amount by a percentage dependent on your initial backing amount. This means the project will still have their funding and you can back with less money!

No overpriced backer 'rewards':
It's everyones choice to back how much they want, but in the regular world, if you make a big investment into a project or product, you can and should expect some of the profits. If you back a project with a lot and then the project ends up earning a lot of money that you wont see at all! As I understands it, Kickstarter won't allow backers to get financial profit projects they support.

I, for one, will be very hesitant to back kickstarter projects in the future. But as I said, it can and should be a great platform for new developers to get a vertical pitch to show for publishers.

Friday, July 26, 2013

ISOC: mink !

Our 4 weeks are up, and here is the result!

- When you are at school and you have problems solving an equation, Wolfram Alpha is a great tool for helping you. However, if you try to use the built-in keyboard of your tablet or smartphone, you will see that it's quite time consuming. Not only that, but if you miss putting correct parenthesis, you will have to wait for wolfram to evaluate your equation before you can try and put those parenthesis into the correct position.
Also, how do you formulate an integration or a sum? This is where our app come into play:


The idea for mink is to simplify the input of complex mathematical equations onto tablets and smartphones in a natural way.

mink - For easy input of mathematical equations and
solve them with Wolfram Alpha


You can also take a picture of an already written
equation, or from a book!


Your Equation is sent and evaluated in
Wolfram Alpha by one press of a button
Save and load you equations for later evaluation.

Restrictions:

We're at the moment, depended on a third-party service for the Optical Character Recognition, which means that we can't  release mink for the public, yet. Since the service is web-based, user can only get character recognition while online. Since we're using 3rd party, there is also some restrictions on what mink can handle.
Right now it can't handle vectors or systems of equations.

Future:

Something we would like to add to mink is to be able to export the equations in TeX format, so it also could be used for simplifying the input of mathematical equations in LaTeX software. 


We'd like to thank IDA for the time and opportunity they've given us to develop mink and also InnovationskontorEtt for letting us use their office during this time.

Monday, July 22, 2013

ISOC week 3: tweaks and fixes

Day 1:

New week, more fixes! We are currently trying to wrap up some work on some odd behaviour with how we switch fragments and how our application handles configuration changes. We also made some progress on the camera module by implementing a thinning algorithm in order to generate one-pixel wide character/symbols. This will hopefully enable us to generate the information we need to send to the MyScript webservice. It would seem like there was some issues with the sliding menu and our canvas data fragment. It was only saved for one configuration change. What we did was to move it out of the canvas fragment and let the main activity handle it instead, which is a better solution since both our camera fragment and future saved and load fragment need to be able to alter this fragment when loading equations.

We've also had more people involved in ISOC show up at the office!
Emil, works on a project he and some of his friends has started called Continuous , you can find his blog here : http://emilaxelsson.se/

Tor's project involves a gamification ala singstar for learning chinese. You can find his project here http://biggestt.github.io/toney/

Day 2: 

A new thinning algorithm (or rather, a slightly modified) was implemented that should guarantee one-pixel wide characters, except at junction points. The work of creating a load and save system was also started, the user will be able to save anything drawn on the canvas and load it at a later stage.

Day 3:

There has been some promising development with the camera module, and it should now properly generate points and pass it to the fragment which handles the canvas and communication with the MyScript service. Turns out that there was just a stupid mistake in a loop which killed off half of the good points. There is still some cleaning up that needs to be performed, and maybe some more filtering on the input image.

The save and loading system is now working and consists of a new menu item with a list of saved equations as well as a textfield and button to save the current canvas.


Coffee, the rocket fuel for programmers


Day 4:

We reached a milestone today where we have most of features working as expected. The thinning algorithm is giving some unexpected results on printed text were lots of characters/symbols have different weights. This could be because of the filtering process, which consists of performing an unsharp mask followed by a bilateral filter in order to remove some off the noise. Then we perform adaptive thresholdning in order to get a binary image. Further clean up is required, so we perform erode and dilation with a 3x3 kernel. Finally, the thinning algorithm is performed (Zhang-Suen).

Day 5:

The previous problems with the filtering/thinning process was caused by the adaptive thresholding. We decided to let the user perform a manual "normal" threshold with the help of a slider and visual feedback. This let us cut away the the unsharp mask and erode/dilation.
So now we have bilateral filter - threshold - thinning, and it seems to work fine for both handwritten and printed text.

Next week which is our "final", will be spent on polishing up the GUI and working on a fluid user experience. Things are looking good!

Friday, July 12, 2013

ISOC week 2: First Draft

Day 1:


Our problems with performance have seem to been solved by Fredrik. He managed to create a well working class so we can transform our gestures in any way we want with a matrix. With this in our bag, We've now started implementing pan and zoom system in the App to give the user even more space and possibility to write their equations. Our UI have been changed into using fragments instead that helps us to store data when user flips their device. This seem to work very well, only issue right now is that we need to load MathJax(Javascript system for displaying LateX Math on a website). It also works great from a developing perspective by splitting the app up to smaller parts that encapsulates the data only important for that fragment.

Our plan is to have a first draft of the app finished this week.

Day 2:

Today we worked with trying to maximize the canvas area for the user to have as much area as possible to draw on. We want to try to hide some part of the UI that is rarely used in a sliding menu. Our intentions is still to have a first draft at the end of the week and it seems like this will be possible.

Day 3:

We've fixed Undo and Redo tools to quickly erase strokes instead of having to switch to the eraser tool.
The first part of maximum and minimum zooming has been implemented. The work of using a sliding menu is still under way but works very good to create more drawing surface.

Day 4:

The min/max zoom system was pretty easy to implement but today, trying to fix min/max panning had it's issues. The thing we we're struggling with was that we, for some reason, tried to keep our origin point of canvas not in the corner, but in an arbitrary point on the canvas. This caused issues when we try to clamp panning in different zoom levels. The simple solution we then found was just to move origin up to the top left corner and move the user down to the middle of canvas when the app starts. We've also added a paper-like background to give a feedback to panning and zooming. The sliding menu was completed and Fredrik started working on using the camera to take a picture of a mathematical expression to draw to our canvas. We think this would be a good solution for people with smaller screens, like phone that perhaps is more difficult to draw on. Since our intentions are to have the expression show up on our canvas, the user can then edit smaller parts if so chooses.
Fredrik reading about Canny Edge Detection
and other cool stuff

Day 5:

The test to parse to wolfram worked quick and simple with no real problems. Wolfram wants the TeX as a urlEncoded string, something we can do easy with android.

We also made our first camera test today, it worked... with questionable results. We are using OpenCV and its findContours method to sample points on our picture, but the result isn't good enough for us to be able to send it for TeX parsing. We are going to look into  different ways of sampling a picture to try to increase our accuracy of the result.

But with that said, we now have what we consider our first functional draft of our app ready!
All the functionality we had when we started the project is now working. There is still a lot of work left though to get a equal experience using the app on different devices. We'd also like to be add features for saving drawings and expressions for reuse.



Monday, July 8, 2013

ISOC 1: Project Startup

Day 1

Today, was our first day working on our project. Most of the day consisted of working out a plan for our data structure and what methods to use for character recognition and also special cases such as, horizontal line a minus or a division operator?

We've been granted to sit at InnovationskontorEtt during our developing time which helps us a lot not losing time commuting to Linköping everyday.

Work Work!


Our first plan is to use the gesture system in the Android SDK(android.gesture) for input as well as character recognition. It's too bad that the resources about the how this system evaluates is scarce since the score seems to be a bit all over the map. Sometimes you get score close to 1 and other time up to 30+.

We've also started to code our nodes we want to use for our data structure and systems for finding relations between these node depending on what type they are as well as their position to one and another.

Day 2

Work continued with our Nodes and also began sketching up the UI and using the canvas to project drawn gestures onto. We've started to look for third-party systems for improving our character recognition since android.gesture doesn't seem to be able to do this work on its own, although it does seem like a good input method since we can do stuff as get the gesture as a grayscale bitmap or a collection of points.

We've also started to discuss how to create and maintain relations between our nodes. Our nodes' children have a variable keeping track on its relation to it's parent, for example: kLeftChild for anything left of an operator or kUpperLimit for the upper limit of a sum or integral.

Day 3

We've continued our work with the UI and found Vision Objects and their MyScript system for recognizing handwritten data. They have a trial online webservice for sending JSON data with sample points to determine what has been written. There was some tinkering to get this to work since the data actually sent wasn't in a JSON format but in an URL encoded method.

While this means we don't really need to further develop our node data structure, we can now put more time into creating an App with user friendliness in mind. We've want to start looking at methods of taking pictures with our devices and generating points to send for evaluation but it will happen in a later stage.

Day 4

The work for the day consisted in trying to minimize the number of queries sent to the MyScript service by letting the user do multiple strokes before any actual data is sent and also threading this query on the app to not lock up the main thread. It seems like the AsynTask in android works very well for stuff like this since it seems to take care of things that could be considered unsafe while threading. While you can tell your app to allow HTTP requests in the main thread, this is something you shouldn't do since it would cause the UI to freeze as well as perhaps reporting that the app is not responding due to waiting for receive a response.

We've also contacted Vision Objects to ask for how much a license would cost and also the possibility of getting a trial period on their SDK.

Day 5

We spent the day in Linköping today. The big topic for today is how we should be able to pan and zoom on our canvas as well as erasing.

A big problem for us right now is how the android SDK handles gesture, gesturestrokes and Paths.
We use gesture for input and a gesture consists of one or multiple gesture strokes. You can then get a path from a stroke, But:
- We can sample points from a gestureStroke, something we need for the MyScript Service. But there is no possibility of transforming the stroke, something we need when zooming.
- We need the Paths to draw on our canvas and we can also make transformations to the path, but we can't use a path to create a stroke and we cant get sample the path to points.

Our current solution right now is to grab the points from the stroke and make our own transformation function but it seems to have an impact on our performance.

Saturday, May 25, 2013

Sky simulation for dome theater




Wow, long time since I've wrote something here, sorry for that. University is keeping me busy :)
One of things I've been working on is this sky simulation for a dome theater at Visualiseringscenter in Norrköping, along with my classmates Aron Tornberg and Sebastian Piwell. It's done with C++, OpenGL and the library SGCT.

SGCT distributes the workload to 6 different nodes, one for every projector. You can read more about it in the link above. I wrote an external GUI in C# for easy tweaking of some parameters.

All the viewports for the dome projection

Fisheye projection
As the screenshots shows, performance is kind of slow on my laptop, but this is work in progress without any optimizations. Besides, the method we use for the clouds is kinda brute force. This probably won't be a problem in the dome theater, since every node renders a relatively small part of the sky.

I'll update/create a new post with more details once we have finished this. I'll try and get some footage from the dome theater as well :)

Additional credit:
Miroslav Andel, for all the support concerning SGCT

Thursday, May 23, 2013

Summer Of Code @ Linköping University

During April, a mail was sent out to all student at our university talking about a new project the university had started: "Summer of Code"(SOC for short). SOC gives students the possibility to develop one of their own projects during the summer and getting payed by the university. It would be  5 projects would be accepted for this opportunity. A total of 70 projects applied, we were one of them and...


Our projects was accepted!


Fredrik and I had discussed an idea for an smartphone app earlier this year that we wanted to create, and decided to apply to SOC with this idea. The final idea, we will keep a secret for now, but I thought we could still discuss parts of it.

None of us have really worked with app-development before, but you have to start somewhere. Our intentions are to develop for the android platform first and perhap, later for iOS since our target audience is students, and a lot of students use iPads. But we also want to keep developing costs down at a start.

The project consists of 3 major parts: Drawing, Detecting, and Evaluating. The biggest challenge for us right now is the second one, Detecting. None of use have worked with image recognition before. We've been looking into using OpenCV as a framework to help us in this stage. The detection should also be "on the fly" while the user is still drawing to give direct feedback to the user.

We are both looking forward to this July when we will start working and we will do some more updates on the blog for each part during development!

Sunday, January 20, 2013

Debugging follow-up: Eclipse

So as a follow up to my previous post, this one is more specific to where you'll find some of the tools of debugging in the IDE Eclipse. Both this and the previous post is targeted to my classmates to help them get started with debugging, but anyone that uses eclipse can check it out!

Lets start off the same place as the previous post, Breakpoints.

In Eclipse there are two simple ways of setting breakpoints. First one is to hit "Ctrl+shift+b". This will create a breakpoint on the current line your typing cursor is.
The other way is to double-click in the white space just left of the line numbers.

You can see in the picture below how there is an indication now of a blue ball on the line you've set your breakpoint on. If you right click this ball and pick breakpoint Properties... you'll get access to edit the conditional breakpoint options I mentioned in the advanced techniques on previous post.

on row 32 we can see the indication that we have a breakpoint set.

So how do we start debugging? Easy, on your toolbar you have these 3 buttons:


The middle one is the regular "build and run", the one to the right is just "run without rebuild".
But it's the one to the left we want to click to start debugging.
When we do this Eclipse tells us that it wants to change "Perspective". When we get this option, we want to press "yes". Now, DON'T PANIC! Your eclipse will change its layout.
So first, I'll tell you how to revert back. In the upper right corner you have buttons for changing the perspective or layout. the one to the very right is our debug perspective, and the one next to it is our regular coding perspective. So a perspective is just another word for "layout to fit what your current doing".

from left to right: open list of all perspective, Java EE persp(never used). Java persp., debugging persp.


We will now have some new windows to look at. One of these is the Variables window, this will show us the variables being changed in the current scope. We can also edit these values here.


If you right click a variable in the editor window of this perspective and pick "watch" you'll get another tab in this window that contains  your "watch list" of variables you want to keep watch on.

So now that we know where to look for our values and we're in debug perspective, we need to control the execution of our code. That's what these buttons in the toolbar are for:
From left to right we have: Step return (same as Step out), Step Over, and Step Into. Read my previous post to figure out what these do. The next one is the Resume button. and the last one to the right is terminate, it will shut down our program and stop debugging.

Lastly, we have a window simply called "debug". This window works as our callstack. So in the picture below we can see that we're right now in the function SetDate from the class Date that's been called from "dateTestProgram.main", the main function of our program and all of this happens in the main thread.


So there we go, you now have everything you will need to start being a more efficient coder!


Friday, January 18, 2013

Debugging tutorial

There is no denying that a good way to learn is to explain to others. That's why I thought I would do this write up of practical debugging tips and techniques.

 What is debugging and why do I need it? 

So, what is debugging? Its the part of software development when you hit the "build" button and you get no errors in the code. Yes, there are no problems with how you've written the code for the computer to interpret it, but what about the logical errors? What about the while-loops that never ends and the if-statments that never returns true? This is what debugging is, resolving your logical errors that cause your software to misbehave or crash.

Easiest way to debug is just put a "print(x);" somewhere in the code to see the value of  x at that point. We've all started out debugging this way. But there comes a time when the logic of the programs we write gets too big, and using this "print debugging" won't be enough. Either your code will be filled with loads of  print-calls to check why your code doesn't enter a if-clause or why the values of our variables doesn't make any sense. This type of debugging is also insufficient since it will never actually tell you where your problem is, just that somewhere after this, something is wrong. That's why you should use debugging tools that many (if not all) better IDEs (Integrated Development Environment) ships with. I will have Visual studios 2008 as a base when talking about most of the parts in this tutorial, but your IDE should have something similar.

Below are some of the key features to use when debugging.

Breakpoints:
One of the main tools you'll be using while debugging is the breakpoints. The name explains its purpose very good; it's a point within your code that will break(or halt) the execution of code. While the code is halted you'll be able to inspect values of variables and, in some cases, even edit them.
Note: The line with the break point has NOT been executed yet.  
Lets look at the code below:
int apples;
apples = 5;
apples = a + 5;

If our breakpoint is set on:
  • Line 1, apples will still not be declared, most (smarter) editors will just move your breakpoint down to line 2 since nothing in your program changes during deceleration.
  • Line 2, apples has been declared but has not been given a value, it will have something like -5468165819 as its value.
  • Line 3, apples has the value of 5.
  • Line 4(not shown), apples will have the value of 10.

But since we don't want to have breakpoints for every line, we have some tools that allow us to "step" in our code.
There are three types of stepping, Step over, Step into, Step out.

Step Over:  The most used one. This basically say "execute the line we're currently on, but halt at the next line".
Step Into: This stepping method allows us to go into functions. If we look at the code below. if we were to have a breakpoint on line 5 and use Step over we would go to line 6 and be able to see the value of ourApples after the function sum has been run. However, if we press Step into, we will jump into the sum function and halt execution on the first line of the function (line 12). We can then continue using Step over to look at every line in this function.
int main()
{
    int myApples = 5;
    int johnsApples = 10;
    int ourApples = Sum(myApples, johnsApples);
    print ourApples;
}

int Sum(int a, int b)
{
    //no need for this, but for showing debugging, we do this.
    int returnSum = a;
    returnSum += b;
    return returnSum;
}
Step Out: If we realize, "The error we're looking for isn't caused in this function", then Step out is used to exit that function and go back to the line that called the function (read more about keeping track of how other functions call functions in call stack part further down).
Let say we have a breakpoint at line 12 in the code above, if we would press Step out we would end up on line 6 again, with no need of stepping over line 14 to go back.

We also have Resume which will continue the execution of code until we hit the next breakpoint.

To set a breakpoint you often press to the left of the editor and some sort of indicator should appear. You can often right-click on a line and pick "set breakpoint" there aswell. There are some other, more advanced topics about breakpoints that you can read further below.

One key feature is the fact that when you hit a breakpoint, you can edit the values of variables! So lets say that right now you're not intersted why your variable's value makes no sense, but you want to try your an if-loop thats within a if-case. If you variable's value is hogwash, the if-case will never be true and you'll never enter the loop - make a breakpoint at the if-case and just change the value of the variable and try out your look!

Breakpoint has been hit, indicated by the yellow arrow, 
By hovering over a we can see that it has a very small value, 
we could draw the conclusion that it has only be declared bu never given a value.

Watch/Local Window:
These two windows help us keep track of our variables without having to hover over them in our editor. In Visual studios, we have both "watch window" and "local window". (some IDEs might not have the "local" one)

The Watch window is at first empty. You have to add variables to it by right clicking them in your editor and click "add watch". Then you'll be able to see the value of that variable as long as it remains in the memory or you remove them, which could be nice if that variable, for some reason, is not close to your current breakpoint.

The Local window shows all variables active in the current scope and therefore changes quite radical when changing scope (when you exit a loop or function).
The local window, the value of the calculate array is indicated by red, 
that means that the last row edited this variable

This way you can see how multiple variables are modified quickly.

Call Stack:
When your programs get more and more complicated and class-functions are called from multiple places, you sometimes what to know "From where is this function called?". This is what the window Call stack can do for you. In the picture below, 
  • The top row shows where our current breakpoint is, its in the function called MoreCalculations on line 39.
  • The next row shows us that MoreCalculations was called from another function, CalculateSum and it was called on line 24 of that function.
  • The rows after that get more complicated but shows us that CalculateSum was called from our windows main and after that we get to even more complicated entries.



But what to take away from this is that we can use this window to keep track of, when and where was this function called, and we can use the call stack to retrace those steps.
Advanced Techniques
Conditional Breakpoints
Sometimes when working with breakpoints within loops, it could be a pain in the butt to having to click resume 45 times, since the error we're looking for occurs on the 45th time the loop runs. That's why we want to use conditional breakpoints. There are often two methods for this.

  • The first one is to tell your editor "I only want to break when i've passed this breakpoint 45 times or more" (could be useful in a while(true) loop)
  • Second one is "I want to halt execution when this variable has the value of 45" (useful in for loops e.g i == 45).
This can be useful to speed our debugging processes up.

Disassembly Code
Just before your code is turned into binary, its first turned into Assembly Code. These are CPU specific instructions that are incredibly cryptic and quite difficult to read if you don't know how. This is a very advanced topic that related more to optimization debugging, but in certain cases, the assembler code can help you find the problem to why your software crashes.
I won't talk about it here, because at the time my knowledge for this is quite limited. But if you're intersted, I'd recommend Alex Darby's Low-level Curriculum series over at #AltDevBlogADay