a_bertrand

Here is my info, if you need more details then ask and I will try to answer: http://makewebgames.io/board721/34508-store-and-retreive-data-when-mysql-is-too-slow BTW spudinski, memory is certainly quick but will fail when your data set is larger than your available memory plus it will be dropped as soon as your process quit or your computer restart / switch off. Also, binary trees in memories are not complex at all to build and offer the same advantage as the one I presented: log2(n) access time instead of n for simple lists (where "n" is the number of records and this returns the worse case scenario).

Problem: There is some cases when MySQL or any pre-made database are simply too slow for what you need to do and at the same time you don't need all the features they offer like you don't need complex queries or deal with different data types. In those cases you can still try to code yourself your own storage engine which is normally not as hard as you may think at first. First of all you must think what you really need. What will you do with your data? Example 1: Create the database once and never edit it again. During the creation of the database you need to be as fast as possible. Example 2: Speed of storage is less important than allowing to add data later on. If you are with the first case, then you should store the data in a flat file and create an index to retrieve it once the storage is finished. In the other case it is much better to store directly with a way to retrieve the data quickly. In my case I choose the option 2. Now how to create an index for the example 1 (as well as it could be used for example 2) ? A good solution would be to create a binary tree with the "key" as node data and position in the flat file. A binary tree is basically a structure where you have some data for the node, and 2 pointers (left, right). If the new value is bigger than the existing one you go to the node right otherwise on the node left. If the node is empty then you add you data there and create the 2 empty pointers. Advantage of this solution over a sorted index is that you NEVER need to have all the keys in memory and therefore can deal with extremely huge data sets. Here you are with a C# code (which should be readable by anyone): static void AddNode(BinaryWriter writer, BinaryReader reader, string key, string data) { // First node? if (reader.BaseStream.Length == 0) { writer.BaseStream.Seek(0, SeekOrigin.Begin); writer.Write(key); writer.Write(0L); // A writer.Write(0L); // B writer.Write(data); return; } // Search a free node in the tree long nodePos = 0; while (true) { reader.BaseStream.Seek(nodePos, SeekOrigin.Begin); string ns = reader.GetString(); long pos = reader.BaseStream.Position; long a = reader.ReadInt64(); long b = reader.ReadInt64(); if (key > ns) // Go to A { if (a == 0) { writer.BaseStream.Seek(pos, SeekOrigin.Begin); break; } else nodePos = a; } else // Go to B { if (b == 0) { writer.BaseStream.Seek(pos + 8, SeekOrigin.Begin); break; } else nodePos = b; } } // Add now the node pointer... writer.Write(reader.BaseStream.Length); // Add the new node at the end writer.BaseStream.Seek(0, SeekOrigin.End); writer.Write(key); writer.Write(0L); // A writer.Write(0L); // B writer.Write(data); }   Binary trees offer quick read / write access normally... beside if you go in the worse case scenario where you create your tree with sorted data in which case the tree sadly de-generate in a list and therefore you have slow access. To solve this you have multiple solutions which could be more or less complex. One of the solution is called red-black trees which basically tries to balance the tree all the time. Honestly I find it a bit complex and not really adequate for a file based solution. My solution is to make a quad tree (4 branches) instead of 2. I use the key and the CRC of the key to find where it goes (left left, left, right, right right):   static void AddNode(BinaryWriter writer, BinaryReader reader, string key, string data) { byte crc = 0; for (int i = 0; i < key.Length; i++) crc = (byte)(crc ^ key[i]); // First node? if (reader.BaseStream.Length == 0) { writer.BaseStream.Seek(0, SeekOrigin.Begin); writer.Write(key); writer.Write(crc); writer.Write(0L); // A writer.Write(0L); // B writer.Write(0L); // C writer.Write(0L); // D writer.Write(data); return; } // Search a free node in the tree long nodePos = 0; while (true) { reader.BaseStream.Seek(nodePos, SeekOrigin.Begin); string ns = reader.GetString(); byte cs = reader.GetByte(); long pos = reader.BaseStream.Position; long a = reader.ReadInt64(); long b = reader.ReadInt64(); long c = reader.ReadInt64(); long d = reader.ReadInt64(); if (key > ns && crc >= cs) // Go to A { if (a == 0) { writer.BaseStream.Seek(pos, SeekOrigin.Begin); break; } else nodePos = a; } else if(key > ns && crc < cs) // Go to B { if (b == 0) { writer.BaseStream.Seek(pos + 8, SeekOrigin.Begin); break; } else nodePos = b; } else if(key < ns && crc >= cs) // Go to C { if (c == 0) { writer.BaseStream.Seek(pos + 8 * 2, SeekOrigin.Begin); break; } else nodePos = c; } else // Go to D { if (d == 0) { writer.BaseStream.Seek(pos + 8 * 3, SeekOrigin.Begin); break; } else nodePos = d; } } // Add now the node pointer... writer.Write(reader.BaseStream.Length); // Add the new node at the end writer.BaseStream.Seek(0, SeekOrigin.End); writer.Write(key); writer.Write(crc); writer.Write(0L); // A writer.Write(0L); // B writer.Write(0L); // C writer.Write(0L); // D writer.Write(data); }   Of course a quad tree takes more space (in memory or disk) but should avoid the degeneration of a binary tree and should offer yet more speed to it. Retrieving data from it is not much more complex: static string GetData(BinaryReader reader, string key) { byte crc = 0; for (int i = 0; i < key.Length; i++) crc = (byte)(crc ^ key[i]); long nodePos = 0; while (true) { reader.BaseStream.Seek(nodePos, SeekOrigin.Begin); string ns = reader.GetString(); byte cs = reader.GetByte(); long a = reader.ReadInt64(); long b = reader.ReadInt64(); long c = reader.ReadInt64(); long d = reader.ReadInt64(); if(ns == key) return reader.GetString(); if (key > ns && crc >= cs) // Go to A nodePos = a; else if(key > ns && crc < cs) // Go to B nodePos = b; else if(key < ns && crc >= cs) // Go to C nodePos = c; else // Go to D nodePos = d; if(nodePos == 0) // Found nothing... quit... return null; } }   Now about performances: Width a flat file (binary or not) I get about 200'000 entries inserted per sec (without counting the index creation at the end). With a quad tree I get about 20'000 entries inserted per sec. With MySQL MyISAM (which is faster than InnoDB in my case) I get about 300 entries inserted per sec. Now to speed things up a little bit more, you could split the files based on the start of the key, if we use a string (lowercase) as key as example we could use the first character to create 26 different files data_a up to data_z, that will speed up to 26x the storage as well as the retrieval. If you use the first 2 characters you will get 676 files with a speed up to 676x faster. In my case, retrieval of items within a data set of 3.3 mil items: 00:00:00.0002379 So it is certainly not slower than MySQL ;) Conclusion: A general purpose database like MySQL or Oracle or any other do have a lot of advantages like flexibility and powerful tools, yet is not optimized for every single cases. In some cases you must create your own storage engine to reach the performances needed by your application and this is certainly possible.

As I'm developping a little toy of mine, mainly for the fun, and during that development I had to face an odd situation where MySQL myIsam nor InnoDB would be fast enough. Yes it's odd but it happens. And yet the situation can be solved by storing things manually without the help of any DB. Yet this storage do require a quick retrieval and within a HUGE number of records. I'm willing to share my solution (pseudo code or C# based) with the community if there is a little interest about it. The basic idea here is VERY quick data storage, then index creation and then the data will be not touched basically and therefore only query from that time on. Who would be interested in such info?

I don't do outsourcing work, so if you need to hire somebody take Lilith ;) Yet, as she said, 3D work is not cheap (either money or time based), so carefully think what you want.

Here you are with 3 different softwares: http://www.blender.org/ http://www.caligari.com/downloads.html http://www.daz3d.com/

"When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. " Means you must keep the sources available and cannot hide them. That's GPL... LGPL on the other side allows to use your "library" or code and the result mix doesn't need to be under GPL nor distributed. So yes, if you use a McCode Lite you must allow people which want to have a copy of your modified version a way to get it. Not sure it is 100% wished for a game ;)

What seems so easy and stupid can be annoying to program correctly. Yet I don't pretend mine fully work but it shows how you could make one: Sources: http://garg.web.nowhere-else.org/web/tutorials/ Demo: http://garg.web.nowhere-else.org/web/tutorials/calc/index.html What's funny with this little calculator is the display which is like old style LCD. I didn't used bitmap for it, it's all vectorial and within the sources.

Hi, I'm trying to come with some good and fun demo to show some of the Silverlight possibilities, as example I create a little particle system which can be downloaded here: http://garg.web.nowhere-else.org/web/tutorials/ and directly tested from your browser here: http://garg.web.nowhere-else.org/web/tutorials/particles/index.html Particles don't collide with each others and don't bounce with objects. However all that could be done if wished. Enjoy!

Will you sell it multiple times or only one? As if it it multiple sales, then 110$ is way too expensive.

They normally have an API you must use to implement the payment and the control of it.

I don't fully agree here. You can certainly start with all free, no domain and no payed hosting and see how it goes. If it works then start by find a cheap hosting but otherwise it would be just money trashed as most of those hobby project end up with nothing...

No I don't have an answer for everything... yet those things I checked quiet carefully so I do have already an good idea of such.

Rasgeed: Carrier fee 49% on Swtzerland, 31% in the UK, 47% for Australia (and those are the average carrier fees) on top of that add 10% fees if you do less than 3500$ per month... Plus it doesn't seems to cover the US... Daopay does it (http://www.daopay.com/)

I personally use DAOPay for SMS and phone payment. Works fine but it is extremely expensive as you get about only 50% of the payment.

Well the idea is not new but at the same time not bad. Now... How will you handle the multi player? or example you do X moves and then you need to wait until tomorrow? Or both players must be connected on the game at the same time? That's exactly the problem with most strategic web games... and where most actually make the things more or less interesting. Beside that, you will need quiet some work like tech research trees, ship design / differences, etc etc... Also, this kind of game MUST be graphically displayed so you need also a good user interface and good art if you want to succeed somehow. Overall it seems a good project and I wish you good luck.

Ok home results are: Yes a huge difference ;) Yet honestly when you browse you see little to nothing.

- If it ends up in the junk folder be happy that my emails allows it, as for example my work email would not work like that and directly trash it without any other option, so no this is certainly not a right answer, sorry. - Email verification and Captcha are NOT both required. Email verification is here if you hope to collect correct emails and that tend to be abused for spam later on, which means it will put off people and you may lose a lot of players. Honestly this is just a personal experience, and you may take it as you want, but you should normally try to make the system user friendly and not push away people. What's the goal? Avoid bots? There is tons of ways to do it. Avoid double accounts? Emails and captcha will not do it... Want to collect emails to spam people? Sorry bad practice...

- Captcha and email verification? => just way too annoying. - email verification OF COURSE ended up in my spam folder. - after validation yet again need to login... - The friend & chat bottom bar doesn't match with the look & feel - lack of tutorial / story at start... you just navigate and click a bit randomly - hints => why not being able to disable them directly by clicking on something on the hint? - hints look is... not consistent with the game - Overall the look is good... but beside that I don't see much a jump from a standard McCode game.

Well it all depends what you are looking for and at which level plus how you personally learn. I learned PHP by simply checking the PHP manual and it was enough for me (but PHP was clearly not my first programming language). Books could be split up in different categories: - Tutorials: which goes through all (or more or less all) the steps to explain you something - Reference: which give you just a list of function / classes with maybe some examples for each - Goal oriented: for example books which take a single example for example a web shop and explain all the steps to make it work - Design oriented: design patterns, algorithms written in the language which you can learn from. So you have to choose what you need / want and maybe once you know it let us know, then we may point you to one more than another.

Indeed did it at work... and we have a fiber connection to "Switch" which is the Swiss Gov / Uni / Research network. So don't expect to have that kind of networks at home ;) Even if at home I do have a good 8-10 Mb download and about 1Mb download.

If you write a PHP script nobody can copy it as they don't have access to the code (unless you make some really stupid things which let the visitors see the code), so no need to worry about the fact that somebody copies your script. Now if you start selling it, it is a totally different story as you can't really prevent that somebody start to distribute your sources after having bought one copy. If you write a javascript on the other hand you have no way to protect it as it is send in clear text to the browser and everyone is free to check it, download it and modify it (same for flash, java applets and other client side technologies where you would need a tool to decompile the thing but you will mostly get the sources out of it).

Well enable the PHP error display in your PHP.INI of your remote computer and you may get more info ;)

Ping test cannot be done from here... Due to the firewall blocking it.

I deleted it as I thought it was starting to be pure flaming and useless.

What exactly do you mean? Change the MySQL version? Then you need to check with those which offer you then hosting. Create tables? Then use the phpmyadmin panel to create, modify your tables.