Archive | Programming

26 March 2018 ~ 0 Comments

Solution: Out of Control MySQL Slow Log Files

I’ve been having an ongoing problem on one of my servers storage space, long story short the culprit ended up being that the mysql-slow.log file was 80GB.

It had gotten turned on for testing at one point and the threshold turned very low and I never thought to check it again over the years.

Anyway, I had a bit of a hassle figuring out how to resolve it so thought I would post it here for future reference.

How to Safely Clear Your Mysql-Slow.log File

This was actually pretty simple once I figured out the quirks.

The first thing to do is to disable slow query logging in mysql:

set global slow_query_logging=OFF

After it is disabled then it will no longer want to write to the file, if it is showing that the file is still in use then simply run:

flush slow logs

After it is disabled you can safely move, compress, or delete the old log file.  If you plan on deleting it then I would recommend at least renaming it first until everything is completed, just as a precaution if the file needs to be restored.

Once you have removed or deleted it, you can create a new one with the same name

>mysql-slow.log

Once it is created, you may need to adjust it’s rights.  This is where I got hung up on, the mysql service was unable to write to the file.

chmod 775 mysql-slow.log

After this is all done, you can turn slow queries back on

set global slow_query_logging=ON

And you will begin slow query logging again in your new file, you can now safely delete the old file if desired.

Hope this helps someone, it’s pretty straight forward but a lot of the other stuff I found when trying to research the problem did not mention having to change the access for the new file and it stumped me for a bit.

Continue Reading

22 March 2018 ~ 0 Comments

ASP.NET Core 2 Not Seeding? Try this!

I had recently created a fairly decently sized application, consisting of about a dozen models, a handful of controllers and a heaping serving of views.  Now, the standard school of thought for seeding data into the database is to create a class in your models and call it in your program startup.  You then have your seeding object check each table for data and insert data as needed.

This has worked for me for the most part, until this project.

I wish I had taken screenshots of the errors, but unfortunately I did not. I was having a lot of problems with concurrency, a second operating started on this project, or that the object has been disposed.

I toyed with it for hours trying to get everything to work properly, sometimes one of the sets of data would seed and other times others would seed, and it even seemed to vary depending on the speed of the system I was using.  I tried breaking everything out into individual methods, awaitng tasks in async methods, and countless re-ordering to try and get the most complete version of the data I could, I even ended up presenting the project with slightly incomplete data as the non-fatal error occurred again during the presentation.  Thankfully nobody knew ahead what I had intended the end result to be, so it went completely unnoticed.  But I knew.

I believe the culprit to be The Async methods required by Identity to create user accounts, roles, and assign roles.  I found the positioning of these seemed to have the most impact on which sets of data would generate errors.

The solution came to me randomly, but is quite simple and could potentially be a better way to seed data than the way I was taught in college and discussed earlier.

It’s a very simple solution, just place your seed data and checks in the constructors for your repositories.   After I split up the data into the repositories the database was seeding flawlessly.  It solved concurrency issues since it was only inserting when the repo object was instantiated rather than all the data nearly simultaneously.

In practice it functions exactly the same way that seeding the data would through a specific seeding class, in that you would have it check the table for entries and then generate them.  Of course, if you don’t need initial data for any of the tables then you won’t have to worry about seeding those tables.

Lastly, as a simple reminder, your seed data will is semi-permanent in an application so it may be worthwhile to make it something meaningful as opposed to just hollow test strings.

Let me know in the comments if this information is useful to you, or if you have anything to add or any other methods for seeding data that myself and other readers may not be familiar with.  Thanks!

 

Continue Reading