<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Posts on Hi! It's me, GPlay97!</title><link>https://gplay97.github.io/Homepage/posts/</link><description>Recent content in Posts on Hi! It's me, GPlay97!</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Sun, 10 Jan 2021 20:29:06 +0100</lastBuildDate><atom:link href="https://gplay97.github.io/Homepage/posts/index.xml" rel="self" type="application/rss+xml"/><item><title>EVNotify - Big PnP Update</title><link>https://gplay97.github.io/Homepage/posts/evnotify-big-pnp-update/</link><pubDate>Sun, 10 Jan 2021 20:29:06 +0100</pubDate><guid>https://gplay97.github.io/Homepage/posts/evnotify-big-pnp-update/</guid><description>I&amp;rsquo;m happy to announce that a big update is finally available for the PlugAndPlay devices of EVNotify.
Actually it is the biggest update ever in the history of PnP.
PlugAndPlay, or shortly PnP, are preconfigured, fully automated devices, that replace the &amp;ldquo;phone in the car&amp;rdquo;. No headaches in forgetting to start the app, no worries about letting the device running - it automatically shuts down and starts. No risks, simply working.</description><content type="html"><![CDATA[<p>I&rsquo;m happy to announce that a big update is finally available for the PlugAndPlay devices of EVNotify.</p>
<p>Actually it is the biggest update ever in the history of PnP.</p>
<p>PlugAndPlay, or shortly PnP, are preconfigured, fully automated devices, that replace the &ldquo;phone in the car&rdquo;. No headaches in forgetting to start the app, no worries about letting the device running - it automatically shuts down and starts. No risks, simply working. Plug it in once. Enjoy always. They are available to buy at <a href="https://shop.evnotify.de">shop.evnotify.de</a>. Very soon also in english.</p>
<p>Are you a proud PnP user? Perfect! Because starting now we are rolling out an update that you waited for such a long time!</p>
<p>This big update contains a lot of bugfixes, improvements and new features.</p>
<p>If you are interested in more details, you can have a look <a href="https://github.com/EVNotify/EVNotiPi/releases/tag/1.1.0">here</a>.</p>
<p>If you update, you will benefit from reliable notifications, bugfixes, and also new data, such as the ODO value from your car.</p>
<p>Updates of PnP needs to be very well tested. Since this was a very big update, it took a lot of time.</p>
<p>Reliability and a high quality standard is a high priority, especially for the PlugAndPlay devices, since one mistake, for example during the update process, can make the device unusable. It&rsquo;s a very high financial risk for me.</p>
<p>That&rsquo;s why we are now rolling out the big update, after a few testers internally confirmed the process.</p>
<p>But keep in mind: It&rsquo;s still a <strong>Pre-Release</strong> update and updating is at your <strong><em>own</em></strong> risk! If you are unsure to update, please wait some time, until it&rsquo;s officially approved for everyone!</p>
<p>Still interested? Here is the current update process. Due to the complexity of the update, the update process differs a bit from the instructions in the manual. It may also change, once it leaves the pre-release state. Don&rsquo;t worry - once this happens, you&rsquo;ll be informed, for example via mail, if this update is recommended for the masses!</p>
<p>If you have questions or troubles, contact info(at)evnotify.de.</p>
<p>Update instructions:</p>
<ul>
<li>choose a location where you are safe and are not disturbed (parking lot for example) - and have GOOD internet connection</li>
<li>turn on your car (such as you would drive - fully on)</li>
<li>keep the car fully on during the whole update process. DO NOT SHUT THE CAR OR PNP DOWN!</li>
<li>Take a phone or any device with a browser and WiFi</li>
<li>Connect to the WiFi of your PnP (SSID and passwords are on the case inside the stick)</li>
<li>Open http://192.168.8.10:3333 from your device, once connected to the PnP</li>
<li>Click on &ldquo;Update Server&rdquo; and wait until process successfully finishes</li>
<li>Wait a couple of seconds to give the service time to restart in the background and reload the website</li>
<li>Now click on &ldquo;Remove Submodule&rdquo; - this is neccessary due to internal changes for the update</li>
<li>Afterwards select &ldquo;1.1.0-prerelease&rdquo; within the version selection</li>
<li>Click on &ldquo;Update client&rdquo; - wait until the process successfully finished (this might take some time)</li>
<li>If you encounter issues, wait one minute and try again. Do not turn off the car or unplug the PnP during the update process</li>
<li>You can check the status at any time via &ldquo;Client status&rdquo;</li>
</ul>
<p>This sounds more complex than it is. <strong>However, if you are unsure please wait for the official release</strong>.</p>
<p>Have fun and enjoy this big update with a lot of improvements!</p>
]]></content></item><item><title>EVNotify - New Year</title><link>https://gplay97.github.io/Homepage/posts/evnotify-new-year/</link><pubDate>Sun, 03 Jan 2021 16:15:16 +0100</pubDate><guid>https://gplay97.github.io/Homepage/posts/evnotify-new-year/</guid><description>There haven&amp;rsquo;t been any new blog posts here for a long time.
A lot has happened lately and this year starts with many changes that I would like to inform you about in this post.
The last year has been a very difficult and chaotic year for each of us.
Also for EVNotify and me as a private person.
This year, too, we will all face new challenges.
So this year starts with a special situation for EVNotify.</description><content type="html"><![CDATA[<p>There haven&rsquo;t been any new blog posts here for a long time.</p>
<p>A lot has happened lately and this year starts with many changes that I would like to inform you about in this post.</p>
<p>The last year has been a very difficult and chaotic year for each of us.</p>
<p>Also for EVNotify and me as a private person.</p>
<p>This year, too, we will all face new challenges.</p>
<p>So this year starts with a special situation for EVNotify.</p>
<p>I had to make some serious decisions.</p>
<p>But first of all, what has happened in the last year?</p>
<p>Well, on the one hand, of course, there were some personal and health problems, which is why no new updates came for a long time.
Nevertheless, a lot of work was done in the background, especially in the last few weeks and months. The increased distribution of EVNotify is of course very positive, but of course it has also presented me with new challenges recently. One of the longest downtimes or limited functionality over a longer period of time recently occurred. A lot had to be done in the background and new, more powerful servers had to be bought in order to guarantee the usual functionality and especially reliability.
Some updates have been released, especially a lot of preparations for future functions. There was also a lot of work on the complete overhaul of the upcoming interface for V3, the next generation of app. Although the new API is not visible to everyone, it represents a notable, important milestone for the development of v3.
Last year I also took over the PlugAndPlay project independently, which was previously done by openWB. This meant a lot of stress for me and presented me with many new challenges, with which I learned a lot. Of course, none of this was easy and, above all, not without risk. I had to pay a large amount of money out of my own pocket in order to be able to realize the project, a completely new version of the PlugAndPlay devices from EVNotify. There were a lot of teething problems, but the more devices built and sold, the more experienced I got and the better the problems that initially existed were fixed.</p>
<p>Whether in the area of PlugAndPlay or the new development of v3, which is in full swing, I was able to learn a lot. 2017 was the year the very first working version of EVNotify was released. A very big milestone for me - and for many people. The beginning of something big was created. Many updates followed. And at some point a completely new generation of app - v2. I learned from past mistakes. V2 isn&rsquo;t perfect either - and v3 won&rsquo;t be perfect either. The important thing is to keep getting better. And that is my claim. To be able to offer you the best.
V3 will not appear overnight. It is important that this is not only functional, but also structurally maintainable. This is important in order to be able to add new functionalities quickly in the future. The past few years, the successes and failures, help make v3 a better and more efficient app generation.</p>
<p>V3 will not only get a lot of great new functions, but also offer new possibilities to continue to guarantee the financing of EVNotify. EVNotify started as a small idea and grew big. So big that I could no longer bear the costs privately and I needed ideas to cover the costs sufficiently. This went very well for a while thanks to voluntary donations, which is why EVNotify has become a sideline of mine. Nonetheless, this was of course not a fair and, above all, stable cost recovery option, which is why I very quickly needed new ideas to secure the financing and thus guarantee a stable and reliable operation. Of course, v3 and all important functions will remain free for the user in the future. Nevertheless, there will be a sustainable concept along with v3 to cover the running costs and the development. The focus is of course on maximum fairness. For example, there will be optional cosmetic functionalities for a one-time surcharge, e.g. the purchase of dashboard themes. But there will also be useful, new functions for an inexpensive membership. Here, too, the focus is on maximum customer friendliness. This means that users will have the opportunity to choose for themselves for which periods they want to purchase the optional membership or not. There will certainly also be the option to purchase a permanent membership for a one-time purchase price. Since not everyone can or would like to afford a membership, there will probably also be the function of looking at advertisements and temporarily activating the exclusive functions of membership.</p>
<p>It also continues with PlugAndPlay. A new, major update will follow in the coming days and weeks for the general public and offers new functions and many bug fixes and optimizations. I would like to thank everyone who has purchased the new PlugAndPlay devices and thus supported the development of EVNotify. When I took over the project from openWB at the time, I had to bear large financial burdens, which unfortunately have not yet been fully covered. Nevertheless, it continues very well. It will take some time before profits can really be made here, not just in the short term.</p>
<p>Through the PlugAndPlay sales and some other income that flows into the secondary business (Twitch, Amazon, programming, &hellip;), the sales limit has been reached in 2020. This means that from this year a lot will change in my accounting. Since the small business regulation no longer applies, I also have to show VAT. What gives me an advantage when purchasing components is unfortunately a disadvantage when selling goods, as I take 19% less in advance. After deducting expenses and taxes, there isn&rsquo;t much left.</p>
<p>So I am now faced with a serious decision. I know that the PlugAndPlay devices may seem expensive, but from a business perspective they are still far too cheap to be profitable in a short period of time. Sufficient reserves must always be available, especially for warranty cases and the constant purchase of new hardware components. Many new insurance policies that have been taken out in the course of this naturally also continuously cost money.
Nevertheless, I know that at the moment, due to Corona, many people are in a financially difficult situation.
I would therefore not like to increase the prices for PlugAndPlay devices.
So I had to decide - raise prices - so that people might get annoyed - or maybe find another way? What about PayPal? PayPal is a very popular form of payment. What many do not know, however, is that they charge very high transaction fees. This becomes very clear even with small amounts. For example, when purchasing a robot that provided voluntary support in the EVNotify web interface, almost 50 ct was deducted from € 2.49 as a fee. Taxes are incurred later on, so that ultimately not much was left. With the PlugAndPlay devices this was just under € 9 or more - just for the transaction fee. For foreign accounts, for example from Switzerland, the fee is even many times higher. Not infrequently, fees of almost € 30 were then due. Of course, this further reduced the already very low profit margin of the PlugAndPlay devices. And what for? PayPal offers real-time transactions. However, this is not relevant for PlugAndPlay devices, because these are manufactured manually by me personally. It does not matter whether the money is credited immediately or on the next working day, as it does not affect the estimated delivery time.
In order not to have to increase the prices, I made the difficult decision to remove PayPal as a means of payment. This is the only way I can offer the same prices. Ordering the PlugAndPlay devices via the website is no longer possible via PayPal - instead there is only one button, as was previously possible, via which an email is sent with a pre-made form, whereupon an invoice is created shortly afterwards , which can then be made by classic bank transfer. I know that this does not offer the same convenience as a PayPal payment method, where you can settle the payment with a few clicks - it not only simplifies the accounting, it also saves time and money. This is the only way to keep the price current. And, and now there is at least one more good news: This is the only way to officially deliver to all EU countries from now on. For a shipping flat rate of € 20, deliveries are now not only available to Germany. This is a big and important step towards getting closer to the goal of bringing EVNotify to the masses. And this step is only possible by removing PayPal as a payment method, because, especially abroad, the fees are simply very high. Even if it is now officially possible via the website <a href="https://shop.evnotify.de">shop.evnotify.de</a>, the international website <a href="https://shop.evnotify.com">shop.evnotify.com</a> will be available in English soon.</p>
<p>Even if this represents a loss of comfort, I still think it&rsquo;s a fair compromise.</p>
<p>This also has an impact on the voluntary support options. Both the robots and the temporary cats in the advent calendar from EVNotify Web can no longer be bought again. Existing purchases will of course remain credited.</p>
<p>This, too, was of course a difficult decision, but one that can only hold the prices through consistent implementation. Nevertheless, the elimination of this possibility of support will of course also result in financial losses. Hence the reference here that there are still many, even free, options to further support the development (more on this at <a href="http://donate.evnotify.de">donate.evnotify.de</a>). Donations via <a href="https://paypal.me/GPlay97">paypal.me/GPlay97</a> are still possible and welcome.
There will also be new possibilities for support in the future.</p>
<p>So what does the near future look like?</p>
<p>On the one hand, a major new update for the PlugAndPlay devices is currently being tested and prepared for the rollout. The reason for the long delays is quality assurance. It is important that the devices work reliably. Mistakes are not only annoying, but in the worst case they are also very, very expensive for me. In order to keep the financial risk as low as possible, the test phase is unfortunately longer than, for example, the app, especially in the case of major changes.</p>
<p>The app also continues vigorously in the background. Some time ago a lot of work was done in the background to prepare the future new API interface for v3. This brings significant improvements and many new functions. This is also designed for many integrations in the future, e.g. for a smart home. This time there will also be complete and good documentation so that you can integrate it yourself.
V3 itself will not only learn from the many mistakes of v1 and v2 and thus make a lot of things better - it will also be based on completely new technology and be completely native. This of course slows down the development at the beginning, since a lot has to be newly developed and conceptualized, but it offers many advantages in terms of functionality and performance and provides the foundation for a lot of exciting updates in the near future. It is planned to start the first internal test phase with v3 very soon. This will initially function as a kind of web app replacement. And promptly also have the opportunity to interact with the OBD2 dongle. The focus here is also on broader support for more dongles - and also on new vehicles. It remains exciting.</p>
<p>I still have a lot of work to do. These are not easy times for me. For us. I can do it. We can do it.</p>
<p>Often times we face difficult decisions in life. You can&rsquo;t please everyone, and I can understand the annoyance of some decisions. However, I would like to offer you the best possible and try to be fair, but I still have to make sure that the reliability can be guaranteed in the future.</p>
<p>I wish you a very good start into the new year 2021. There is a lot to come and it will be great. Let&rsquo;s look forward to the good times, even if the hard times are still ahead. Thank you for your understanding and I hope to give you a lot of detailed information about new updates soon.</p>
]]></content></item><item><title>EVNotify - Birthday</title><link>https://gplay97.github.io/Homepage/posts/evnotify-birthday/</link><pubDate>Fri, 19 Jul 2019 21:22:34 +0200</pubDate><guid>https://gplay97.github.io/Homepage/posts/evnotify-birthday/</guid><description>More than 2 years ago - the 4th of Juli 2017, the very first official version of EVNotify was available for the public.
Even when the first version was very basic - compared to the current version - it took months of active development to create it. It was a hard time and I was nearly giving up.
But then, many of you, encouraged me to try harder - and I finally made it.</description><content type="html"><![CDATA[<p>More than 2 years ago - the 4th of Juli 2017, the very first official version of EVNotify was available for the public.</p>
<p>Even when the first version was very basic - compared to the current version - it took months of active development to create it. It was a hard time and I was nearly giving up.</p>
<p>But then, many of you, encouraged me to try harder - and I finally made it.</p>
<p>This was not only something - it was the beginning of something huge.</p>
<p>EVNotify was originally created by me because I wanted to solve the problem, that the Hyundai IONIQ Electric does not come with an app here in Germany to allow remote monitoring of your state of charge.</p>
<p>I read about many people complaining about this, but no one really tried to do something to solve this issue.</p>
<p>So even before I decided to really buy an own Ioniq, I decided to try to solve this issue.</p>
<p>It was a very long and hard time. At the beginning of the development of EVNotify I was only 19 years old and just started my apprenticeship as a software developer.</p>
<p>I never thought that I could make this - and especially I never thought about, that EVNotify would be such as successful application.</p>
<p>More and more people used EVNotify and I added more and more features over the time.</p>
<p>Currently, there are several projects related to EVNotify. The server, the app, the web interface, the PlugAndPlay version, Python version, and more to come.</p>
<p>I created and still maintaining it in my free time. For free. Open-Source.</p>
<p>It was my goal to create something useful, that everybody can use. It was never my goal to make money out of it.</p>
<p>However, EVNotify has grown a lot since this. Even me.</p>
<p>Today is my birthday. I&rsquo;m now 22 years old 🎉</p>
<p>I learned a lot from EVNotify. And that is way more worth than any money.</p>
<p>Day by day people are using EVNotify to make their daily life easier. This makes me really happy.</p>
<p>Currently, EVNotify has more than 3.500 registered users and it&rsquo;s growing everyday.</p>
<p>At the early stages, I was happy, when I saw a request incoming, manually logged via console logging statements. If I would have this still today, the server would explode, I guess ;)</p>
<p>In average, EVNotify processes more than 60 requests per second! 24 / 7.</p>
<p>At peak times, these increase to several hundreds!</p>
<p>Even when I&rsquo;m really happy to see, how many of you are using it, it created new problems.</p>
<p>I had to adjust the server architecture to still be able to process all this data and requests.
It was very challenging but I made it.</p>
<p>Since the very beginning I payed everything by my own. All the server costs, licenses, hardware costs. This is a lot of money. But every cent is well invested.</p>
<p>I learned so much during coding this project. Via Slack or other channels I created a very friendly community and today, even others are contributing to the source code of EVNotify.</p>
<p>In these 2 years, so many features has been introduced, so many will be added very soon. It would be to long, to list them all here.</p>
<p>But just a reminder:
EVNotify started as an idea. With one commit. One data, the state of charged, that can be read out. One car that was supported. One button that manually activated the monitoring.</p>
<p>Today, it has nearly 2.000 commits across all the repositories, nearly 20 different data fields, that are extracted, nearly 10 different supported cars and different mechanisms to watch and handle the monitoring automatically.</p>
<p>This would not be possible without you! Everyone that is using EVNotify. Everyone that encouraged me to develop and maintain EVNotify, even in my free time. And it would not be possible without <a href="https://github.com/EVNotify/EVNotify/wiki/Donation">donations</a>.</p>
<p>Remember: There is no free product. Even when it&rsquo;s free.</p>
]]></content></item><item><title>EVNotify - iOS Bluetooth</title><link>https://gplay97.github.io/Homepage/posts/evnotify-ios-bluetooth/</link><pubDate>Sat, 06 Jul 2019 13:21:09 +0200</pubDate><guid>https://gplay97.github.io/Homepage/posts/evnotify-ios-bluetooth/</guid><description>EVNotify is a hybrid mobile application which means that the main code itself is platform-independent. On top of that native code is additionally added to interact and communicate with native, platform-specific functions.
Nevertheless, EVNotify was for a long time only available on Android. This had several reasons. One major reason was, that EVNotify is an Open-Source project built and actively developed by me in my free time. For free. So I earn no money.</description><content type="html"><![CDATA[<p>EVNotify is a hybrid mobile application which means that the main code itself is platform-independent.
On top of that native code is additionally added to interact and communicate with native, platform-specific functions.</p>
<p>Nevertheless, EVNotify was for a long time only available on Android. This had several reasons.
One major reason was, that EVNotify is an Open-Source project built and actively developed by me in my free time. For free. So I earn no money. Developing apps for iOS requires an Apple devices, such as a MacBook. This costs of course a lot of money. Furthermore, the iOS license is, compared to the Google license, very expensive.</p>
<p>While it costs only once 25$ for Android, Apple wants 99$ for the license - per year. That is a lot. Furthermore developing Android apps can be done on every operating system and does not require proprietary hardware.</p>
<p>Fortunately, a very friendly user of EVNotify, that lives near me, rent me a MacBook Air for a while (now I also bought a 2015 MacBook Pro by myself). With this, I &ldquo;only&rdquo; had to bought the iOS license.</p>
<p>The very first iOS version of EVNotify was born, a few months ago. It was a huge milestone for me, because it also required several modifications to the code and it was a complete new experience dealing with the Apple Review process. It was really hard, because so many things were rejected. Every app and update will be reviewed by Apple from real humans. Every review takes many days to finish. And every patch to this, you submit, starts this process from the very beginning.</p>
<p>But what was the problem? If you are using EVNotify on an iOS device, you already faced this issue.
The app was only useful as a receiver. It could only receive the data, the Android device with the EVNotify app submitted.</p>
<p>But why you might be asking? This is because the Bluetooth handling for iOS was not working.</p>
<p>Unfortunately, Apple decided to not support the serial, classic Bluetooth protocol, all the OBD2 Dongles, that are currently working with EVNotify, are using.
Apple only supports Bluetooth Low Energy devices. Technically they work completely different and this is the reason, why the Bluetooth communication with the Dongle, even with - in theory - compatible BLE Dongles, didn&rsquo;t work.</p>
<p>I tried to get this functional. I spent many hours trying to figure out, how to implement this different protocol. Trying to understand, how BLE works, especially all the characterics, they are using, to retrieve, write and listen to data.</p>
<p>Due to several other pending todos in EVNotify, the iOS version was never updated. And the bluetooth handling never worked.</p>
<p>A few days ago I tried it again. And spent several hours again. Yesterday I jumped in the car after my normal work and focused to get this functional.</p>
<pre tabindex="0"><code>./activateHackerMode.sh
&gt; Hacker Mode activated
</code></pre><p>It looked really funny, I sat in the car for over an hour with the MacBook on my legs with several applications opened, the IDE, terminals, &hellip; Connected to a physical iPad in debug mode.</p>
<p><img src="/img/IMG_20190705_193112.jpg" alt="Sitting in the car"></p>
<p>Finally&hellip;. After another hour .. The dongle suddenly flashes. And again. And again. And the cycle displaying the state of charge of the car showed the real state of charge.</p>
<p>It worked. It finally worked! 🎉🎉🎉🎉🎉🎉</p>
<p>What a milestone!</p>
<p><img src="/img/IMG_20190705_195322.jpg" alt="It worked"></p>
<p>What does this mean for you now?</p>
<p>If you are an iOS user and you don&rsquo;t want or don&rsquo;t have an Android device, but for example an old iPhone you are not using anymore, you can use EVNotify now as a sender!</p>
<p>When exactly? In the next update. Stay tuned. Due to the manual review process of Apple and a few modifications I think it will be live somewhere next week.</p>
<p>Follow EVNotify on <a href="https://twitter.com/EVNotify">Twitter</a> and stay always up to date!</p>
<p>Important: The recommended dongle for Android will not work with iOS. You need specific ones, please use the one (tested) listed <a href="https://github.com/EVNotify/EVNotify/wiki/Compatibility">here</a>.</p>
]]></content></item><item><title>EVNotify - Logs Migration</title><link>https://gplay97.github.io/Homepage/posts/logs-migration/</link><pubDate>Tue, 02 Jul 2019 20:58:59 +0200</pubDate><guid>https://gplay97.github.io/Homepage/posts/logs-migration/</guid><description>EVNotify has a quite useful and powerful feature, users are using daily. Logs. With logs, all your charging sessions and optionally even your drives, you monitor with EVNotify, are summarized for subsequent inspection.
All the data, you are collecting during your session, such as battery temperature, the current speed you are currently charging, the state of charge, and many more data, are then visible and rendered in a chart. If you have location syncronization enabled, you can even see a map, where you drove.</description><content type="html"><![CDATA[<p>EVNotify has a quite useful and powerful feature, users are using daily.
Logs.
With logs, all your charging sessions and optionally even your drives, you monitor with EVNotify, are summarized for subsequent inspection.</p>
<p>All the data, you are collecting during your session, such as battery temperature, the current speed you are currently charging, the state of charge, and many more data, are then visible and rendered in a chart. If you have location syncronization enabled, you can even see a map, where you drove.</p>
<p>To be able to provide this functionality, the server needs to process your data, after your session ended.</p>
<p>This is done in a quite complex way, because the data is highly dynamic and is not pushed in a steadily period. It differs for several reasons, such as bad client internet connection, local binary data reading and analyzation from the car, or even the server itself, when it&rsquo;s overloaded. This is the reason, why the data will then be summarized afterwards on the backend.</p>
<p>Usually, it takes about 5-10 minutes after your session ended, until the log is visible within the app or the brand new <a href="https://web.evnotify.de">web interface</a>.</p>
<p>Now comes the interesting part. Why does it take that long? Well. On the one hand, a delay of 2-3 minutes is normal and wanted. Because, as mentioned earlier, you have to keep in mind, that connection can be lost for a few seconds or a minute. With a grace time of 2-3 minutes of &ldquo;no data is coming anymore&rdquo;, you ensure, that the session has really ended.
On the other hand all the submitted data needs to be processed. And this is a really cpu-intensive thing. Because it needs to retrieve all the data has been submitted since the latest log, that has been stored on the database. This can be a lot of data. Actually, it&rsquo;s up to several thousands every few minutes, when a new log is generated. Every sync will be stored, so every data, like the current state of charge will be transmitted, so you can afterwards see a chart with the history, how it evolved. But, and that is producing the most traffic, also location data is included. When you drive your trip with EVNotify and location syncronization turned on, you are submitting location data every second or even multiple times per seconds, based on the precision and the current speed. Imagine, how much data will be sent to the server, when hundreds of users are using EVNotify on a daily basis.</p>
<p>So all of these data are logically combined and then merged and summarized to a log entry. This log entry just defines, which user when drove or charge at which time.</p>
<p>You can not really avoid the huge calculation that is required to logically summarize the drive or charge. And even the huge amounts of data that needs to be processed. Because this is a very cool feature, that will be even more extended in the future to also be able to track consumption and the amount of energy you have charged (new post will follow soon).</p>
<p>But the problem that occured is, that during the existence of EVNotify v2, that introduced this log feature, all data are stored there. Over time this amount of submitted data increased exponentially. Currently more than 50 million data has been submitted at all, which is so amazing and unbelievable!</p>
<p>But now you get the problem. It&rsquo;s a lot of data. Like, really, a lot. You can imagine the database power, that is required, to prefetch and query all these data to then select only the required rows, that needs to be processed the next.</p>
<p>This is not the only problem. Because, if the generation takes 5 minutes or 6 minutes, it is not a big deal for the user. But if you want to open your log and you need to wait like 20 seconds instead of 1 or 2, this is really frustrating and not a good user experience. Especially for drives, the loading time was not good in the last few days and weeks.</p>
<p>So I needed to really think about, how I can improve this. A few weeks ago, I upgraded the server hardware to provide more power. This, of course, helped a little bit, but it&rsquo;s not solving the problem and just delays the real problem.</p>
<p>I came up with a new idea, that helps a bit. Migration of the logs. Instead of storing every &ldquo;statistics&rdquo; / history data (data you submit) in a single, huge table, it will be splitted monthly in own, dedicated ones. With that, the amount of data to load and process is divided by 12. Having less data, optimizes database operations a lot, so the real effect and performance boots is even more.</p>
<p>On the weekend I thought a lot about how I can achieve this. Especially during the live execution with no downtime. Users should be able to still use EVNotify. They should not even notice, that their logs are processed and migrated in the background.</p>
<p>This is a real challenge, especially for the server of EVNotify. It&rsquo;s not the biggest one, EVNotify is an Open-Source-project, that I build and develop for free, so I don&rsquo;t have a fixed or reliable income from EVNotify. I rely on <a href="https://github.com/EVNotify/EVNotify/wiki/Donation">donations</a> and my own pocket bag ;-)</p>
<p>So, the current solution is, to retrieve existing logs, move the associated data from the &ldquo;big table&rdquo; to the corresponding &ldquo;monthly table&rdquo;. In future, it may be changed to weekly, but this needs to be done later. The migration takes a lot of time and is really, really, computationally intensive.</p>
<p>If you are interested in the code, you can look at the <a href="https://github.com/EVNotify/EVNotifyBackend/commit/e43bef7e669327f7bdeaaec4fc3c8b214fc9b0bb">commit</a>.</p>
<p>Currently, about 33,51% logs are already migrated. Several thousands are still migrating. The migration started about two days ago, so the migration itself will take at least two or three more days, until every log is migrated.</p>
<p>At the end, to give you an example of the performance boost of a driving log of 23 minutes.
Before the migration, it took about 5.6 seconds until the data has been retrieved.
Now, after the migration, the time decreased to less then 200ms.</p>
<p>I hope you liked this very first blog post, especially the insight of the EVNotify migration. I will continue writing more blogs to give you updates about what I&rsquo;m doing and what I&rsquo;m currently working on.</p>
<p>Thank you.</p>
]]></content></item></channel></rss>