Query-1: SQL Server log file growing unexpectedly?

“We are wondering why transaction logs keep on increasing?? It become so gigantic, the transaction log size get increase from 135GB to 350GB and that’s so annoying. My drive containing data store is losing space. I am using Full Recovery Model and the logs are backed up hourly. Please help me out why this is happening and what should be done to stop growing it too big?”

SOLVED: SQL Server Log File Getting Too Big

SQL Server Transaction logs records all activities of transaction logs. Each database in SQL Server instance consists of logs that records all the modification done in the databases. It is crucial component of the databases. Any modification, deletion can be tracked down in transaction logs.

There are various reasons of transaction log file getting expanded. The possible reasons are listed below:

• Large transaction log file
• Transaction taking too long to complete
• Transaction operation fail & may start to rollback
• Issue in performance
• Blocking issue
• Database participating in AlwaysOn availability group

What to do for stopping growing log file too big

1. Change the type of Recovery Model to Simple Recovery Model only if you are fine with not able to restore logs hence, failed point in time recovery. It is because if you are truncating your logs, you are breaking Transaction log LSN & if any disaster occur, you will not able to restore T-logs. Hence if you are fine with this situation, then change your Recovery model to Simple. This will not allow any extra growth of log file.
2. Take T-log backup every hour in Full recovery Model, your transaction log will keep on growing until you take backup and truncate of it. While taking hourly T-log backup, the logs will grow until one hour, but after this it would truncate all commited transaction.

Sometimes it might happen that you are in Simple Recovery Model, but your logs are still growing!!! Reason being is long running transaction. If your transaction taking long time when deleting thousand records in one delete statement, then logs will not be able to truncate until delete operation is done.

You can save yourself by maintaining proper size of log file or monitoring the usage of transaction.

By doing this, you can cut down the size of Transaction log.

Query-2 How To Shrink Transaction logs File in SQL Server?

“My log file size is growing continuously. Why is this happening?? I am lacking my disk space. I want to shrink my Transaction log file. Can anyone tell me how to shrink transaction logs in SQL Server. What will be the after effects if I shrink the log file.

SOLVED: Reduce SQL Server Database Transaction Log File

Relax!! There are several reasons for transaction log file growth. Long Running transaction, lack of taking log backup are some of the reasons which result in transaction log growth. Databases running for business purposes or production requires appropriate recovery model to be chosen for log management.

Log truncation frees up the log file space so that logs can reuse it. Truncation occur automatically if the database is backed up in Simple Recovery Model or after log backup when database is in Full Recovery Model.

Here I will resolve your shrink issue by two methods: By using SQL Server Management Studio and by using T-SQL.

Shrink By SQL Server Management Studio:

• Right Click on your Database–>Tasks–>Shrink—>Files.
• You will get the Shrink File Window, Change the File Type to Log.
• There are three different actions under Shrink Action option. Either release the unused space, or shrink file to reorganize pages before releasing unused space, or empty file by migrating the data to other files in the same filegroup. Choose it according to your need.

Shrink By T-SQL:

You can simply shrink transaction log file by running DBCC SHRINKFILE after taking log back. If you are using Simple Recovery Model, run below code:

DBCC SHRINKFILE (TestDB_log, 1)

If your database is in Full Recovery Model, first of all, set it to Simple Recovery Model, Then run DBCC SHRINKFILE and then set it to full.

Caution: Setting database again in Full Recovery Model leads to loss of data in the Log. So, if you dont care about losing your log data, then definitely go for it!

Tips when you plan to shrink database or file:

• Shrink operation found to be most effective after delete or drop table operation as it create lot of unused space.
• If you are repeatedly shrinking your database and you notice that the size of database grows again. Alert!! Shrinking database is a wasted operation, as most database require free space for regular day-to-day operation. This indicates that the space that was shrunk was a part of regular operation.
• You should not shrink your database or data file after rebuilding indexes. As shrinking increases fragmentation to extreme level.
• Do not ON Auto_Shrink option. (Unless you have very urgent requirement)

Is Shrinking database Bad?

Well, Yes.

After shrinking operation,you will definitely be able to reduce the size of the database. But Wait!! Check your Fragmentation level first. It is found to be way too much!!!

Shrink operation increases the value of the fragmentation way too much. And higher fragmentation costs you poor performance of the database.

While if you are thinking that you can reduce fragmentation by using rebuild index, let me alert you, this will definitely reduce your fragmentation level. But Let me check the size of the database The database size increases way higher than the original.

Query-3 How to check Deleted Records in SQL Server?

“I am working in an organization and my manager assigned me work to check deleted database records of my database. How can I check my deleted records in SQL Server? Please Help me out from this situation.”

SOLVED: Ways to Check Deleted Records in SQL Server

You can track down your deleted database records by reading Transaction logs. And you can read it by using fn_dblog function. You can track down any DML as well as DDL activity like insert, delete , update, create operation by checking your transaction logs.

Fn_dblog function enables users to read all the database activity happening in SQL database. The function require begin and end LSN number for a transaction. You can easily trackdown the information of a person who did changes to your database. As LSN number are not in human readable form, SQL Server provide fn_dblog function for easy reading of transaction logs.

You can simply check your database activity by implementing below code:

USE ReadingDBLog;
GO
select [Current LSN],
[Operation],
[Transaction Name],
[Transaction ID],
[Transaction SID],
[SPID],
[Begin Time]
FROM fn_dblog(null,null)

How to Read Transaction Log Quickly

One quick way to read SQL Server database activity is to use SysTools SQL Log Viewer. It read & analyse operation like insert, update, delete. The software previews all .ldf activities like transaction name, table name, query etc. You have option to choose the database from different mode ie Online & Offline mode. You will able to fetch it from live databases. If your database is in Simple Recovery Mode, the tool is capable to recover deleted database records. You will get three different option for exporting. You can export it as in SQL Server Database , as CSV or as Compatible Script.

Conclusion

The blog covers various users queries about transaction logs along with its possible solution. I have discussed why transaction logs keep on increasing, how to track down your transaction details of your database and how to shrink transaction logs. I have also discussed some quick tips when you shrink your database.

Problem:

A few days ago, while migrating data from one database to another, I encountered an interesting problem which user faced a lot.

“I have two databases, I have to copy tables data from one database to another. My query is how can I copy table schema from one database to another with its schema and data”

In this blog, we will learn the easiest solution on how you can solve that query.

Solution

There are plenty of methods on how you can copy table schema and data from one database to another like Generate script wizard, Import/Export Wizard etc. But these methods are quite lengthy and requires users time.

The easiest solution to copy schema’s table from one database to another is to use SysTools SQL Server Database Migration Tool. You will able to migrate your tables schema by performing few steps.

How to Copy Table Schema and Data from One Database to Another?

• Launch SysTools SQL Server Database Migration Tool.



• Click Open and load your database MDF file.



• Choose the Scan Modes accordingly and select the SQL Server version of your .mdf file.



• Select the database table from the left side, which you want to migrate to another database.



• Click on Export for migration process.



• You will get two options for exporting. One is SQL Server Database and other is SQL Server Compatible Script. Choose it according to your need.



• After filling the Server credentials, Select the destination database as ‘Export to Existing database’



• Software will fetch the databases and let you select the database on which you want to move the selected table.



• You will get two options to export. Select ‘With Schema & Data’. This option will copy schema & data of your selected table from one database and export it to another and click Export.



• You will get popup of Export complete.

Conclusion

The blog covers the easiest workaround of mostly faced users query on how to copy schema & data from one database to another. You can also use Import/Export Wizard, Generate script wizard etc for object migration using SQL Server. But these methods has some limitations. In Import/Export wizard, the indexes and the key will not get transferred, you need to generate scripts for that. The export wizard fails if foreign keys connecting the tables together is not in a correct order. ‘Generate script’ method is used to generate a single script for tables schema, data, indexes and keys. But again, the limitation is this method doesn’t generate tables creation script in correct order.

So one can opt for SysTools SQL Database Migration Tool for fast migration of tables schema from one database to another.

Introduction

Structured Query Language (commonly known as SQL) is a programming language used for managing data held in relational database management system (RDMS) consisting of data definition language, data manipulation language and a data control language. The SQL database comprises of following three files:

1. Primary Database Files: Primary database file is the main database file (MDF) which points to another file in database and hence every database has one primary data file. The file is in the extension of .mdf
2. Secondary Database Files: When the data of a database exceeds than a secondary database file is created which stores the data that exceeds MDF limit. Hence, multiple secondary data files can be created for a single database. The file is in the extension of .ndf
3. Log Files: These files maintain a log of all the transactions done in SQL Server Database so that that information can be later used to recover the database. There must exist one log file for each database and it is possible that more than one log file can be created for a single database. The file is in .ldf file extension.

Log files (also known as transaction log) consist of actions executed on the database for database management to guarantee ACID properties over crashes or hardware failure. It is a file listing changes to the database, stored in a stable storage format

Problem

The log files of the SQL Server gets corrupted i.e. while the time we are performing some action upon SQL database, there exist some errors in between the process and due to which there is an interrupt in process.

Causes of Log File Corruption in SQL Server

• Viruses or other Malicious Software- In computer system, many viruses can infect and damage the log files and makes them inaccessible.
• Terminating System Abnormally- If system/application is quit abnormally, then files are prone to be corrupted or damaged.
• Input Output Configuration- The I/O subsystem is a vital component of the database used to store system and user databases. Hence if the configuration is disturbed on enhanced that it may lead to corruption in log files.
• Storage Size Issue- The biggest reason behind the corruption of log files is the storage size. In case the data exceeds the limit of LDF, corruption is likely to occur.

Errors which Occurs Due to Log File Corruption

Error Message 1: StartLogFiles: The error exists when the log files are unable to start because the system could not find the file specified. So try to diagnose and correct the operating system error, and retry the operation.

Error Message 2: File activation failure. The error occurs due to error in file on location ‘C:\ProgramFiles\MSSQLServer\MSSQL10_50.SQLEXPRESS\MSSQL\Log\ERRORLOG’.

Error Message 3: The error message displays that the transaction log has been deleted or lost due to a hardware failure of a system or any other reason. The log files cannot be redeveloped because users perform open transactions on file.

Error Message 4: Corrupted server of SQL database leads to corruption in backup of log files.

Error Message 5: When database log is corrupted and a user is attempting to attach the log file to the new server then an error message displays with a message that ‘Could not open new database (Name of Database). CREATE DATABASE is aborted.

Error Message 6: When the log database is attempted to attach but it gives an error while performing the attachment. The error displays one of the two number i.e. 9004 or 9001 notifying that you have to create a backup or it is necessary to rebuild the log.

How to Recover Corrupted Log File in SQL Server?

In order to repair a corrupt LDF file, use the WITH TABLOCK option for DBCC CHECKDB. It will recover the data from a corrupted LDF file that has been corrupted or damaged due to some reason such as logical corruption.

dbcc-checkdb

Conclusion

Through the above solution, we can recover SQL log file but not sure that the above procedure will recover complete log file data. However, there exists SQL Server transaction log recovery tool through which you can restore database transaction log.

To get a damaged MDF file is not rare. However, the main cause behind it is that the file is prone to corruption. When this occurs, all data saved in the server database becomes inaccessible and therefore, it leads to data loss. In such circumstances, it is important to repair corrupt MDF file of server database storage.

This segment will discuss how to execute the repair process. In addition, it will also give you in-depth information about MDF file, reasons, and way to repair damaged MDF file to store it in healthy form. Read further to know more.

Quick Glance on MDF File

SQL Server MDF file is a relational management system of the database. Moreover, it is a primary data file of the database, which saves all the server data. Thus, you can even state it as the master database file of MS SQL Server. Every database of SQL Server would enclose at least one .mdf file.

It saves components like XML Indexes, Views, Indexes, Stored Procedures, Tables, Triggers, Rules, Keys, User Defined Functions, sparse columns, data types, file stream, the column set property data types. The .mdf file can be said as a primary element for managing SQL Server database.

Reasons of MDF File Corruption

There are various causes, which are answerable for damaging the primary file type of server.

• Unexpected power failure.
• Various bugs in server.
• Defective Operating System.
• Sudden shutdown of the machine.
• Issues with hard drive
• Virus outbreaks.

Thus, issues for your .mdf file revolving corrupt can be everything from malfunctioning of hardware to software and so it is necessary to fix corrupted MDF file.

Technique to Repair Corrupt MDF File

Method 1: Inbuilt Tool

There are some tools that make easy for users to fix corrupted MDF file of the server. Thus, make the saved data available. In fact, all these tools are series of commands in T-SQL programming language, which is called DBCC (Database Console Commands). The purpose of these statements in DBCC is to test physical and logical uniformity of MS SQL Server database files and fix troubling problems that continue.

DBCC CHECKDB is a command via which you can simply check the logical and physical integrity of all objects in precise MS SQL Server database. You can do it by implementing the mentioned operations successively:

• Run DBCC CHECKALLOC command in the database.
• Run DBCC CHECKCATALOG command in the database.
• Run DBCC CHECKTABLE on every view & table in the database.
• Verifying content of every indexed view present in the database.
• Validating link-level constancy among table Metadata, file system directories files while saving varbinary (max) data utilizing FILESTREAM in the file system.
• Confirming Service Broker data in database.

Once the above steps to repair damaged MDF file is completed then, if the application finds any issues of corruption or any errors, it recommends users have the usage of various recover options for fixing troublesome problems. The recover or repair options are:

• REPAIR_FAST

It preserves syntax for compatibility of backward only; no repair actions are executed in definite. The syntax for this, Repair option is: DBCC CHECKDB (‘DB Name’, REPAIR_FAST).

• REPAIR_REBUILD

This option of repair implements repair process, which scarcely has potentials of data loss. This can do quick repairs like repair of missing rows in non-clustered indexes, even time-consuming repairs like the rebuilding of indexes. The syntax is DBCC CHECKDB (‘DB Name’, REPAIR_REBUILD).

• REPAIR_ALLOW_DATA_LOSS

This command creates an effort to fix all the issues, which are reported. However, it can root cause the loss of data as specified in repair command itself. The syntax is: DBCC CHECKDB (‘DB Name’, REPAIR_ALLOW_DATA_LOSS).

Limitations:

1. The specific database should be in single-user mode to be able to execute either of three commands of repair.
2. DBCC repair commands are not authenticated for memory-optimized tables.

Method 2: Trouble-Free Solution

There can be a possibility that when your SQL Server database MDF file is corrupt and when you try to connect to SQL Server you will find that it is marked as SUSPECT. During such scenarios or the above discussed, you will not be able to connect to the database. To repair corrupt SQL Server MDF database file best option is to restore from a recent full database backup available.

If no recent backup available in such cases best possible approach is to repair corrupted MDF file of Microsoft SQL Server 2000, 2005, 2008, 2012 and 2016 with SQL Database Repair Tool. One of the most highly used and recommended software to repair corrupt SQL MDF file with the assurance of 99% guaranteed recovery.

Observational Verdict

We have gone through the possible solution which you need to follow to recover a database in case MDF file get corrupted or database marked as SUSPECT. We serve you with the best of the best ways to repair corrupt MDF file. So, Hurry! What are you waiting for? You are raising the bar and we are taking it forward with the all in one and probably the best of solution repair damaged MDF database file.

Database Files and its Storage Structure in SQL Server

Database in SQL Server has two operating system files:

• Data Files
• Log File

Data Files

A data file consists of objects and data. Depending upon the needs, data files can be divided into two types:

Primary Database File

Each database has one primary data file. All the tables, database objects, indexes and views are stored in that file. The file extension of primary database file is .mdf.

Secondary Database File

This type is used at the time of exceeding the maximum allocated size for an individual file on Windows. It helps the database to increase in size. File extension for this file is .ndf.

Log File

Whenever users want to recover whole of the database in SQL Server, they require certain information, which are inside a log file. For one database there will be one log file and transactions of database are written on it even before to the data file. The file extension used by it is .ldf. LDF stands for Transactional Log file.

Inner Structure of Data File

Each database has its space to store the data, which is divided into different pages that are numbered from 0 to N. During the expansion of database files from its default size, those pages which are created newly are numbered from the last highest page number plus 1. In the similar way, at the time of shrinking of files it eliminates pages in decreasing order, which in the same manner starts from the highest page number. The basic unit of I/O for SQL Server operations is a page and each page is of 8 KB. A data file is a large array of pages. SQL Server stores different data with the help of multiple pages, Some of them are GAM, Index, IAM, Data, SGAM, and TextMix.

Page Addressing:

Now, if we analyze the pages addressing then we can see that every page in SQL Server is of equal size that is 8192 bytes. Each page has its own address, which is unique. It is a part of single database file. The ID of the file makes the first part of the unique address. Pages are numbered starting from 0. This is the mentioned format in which page address is written:

file number : page number

Data page is very essential among the page types. This is used for the storage of records to a database.

Structure of Data Page:

A data page contains three sections.

• Page Header
• Page Body
• Slot Array or Row offset

Page Header (0-95 bytes)

The first 96 bytes of a particular page is known as page header, which consists of the information of the page.

That is, Page ID, No of records in that page and IDs of previous/next pages.

Page Body

It is that part where actual data is stored. It is followed by the free space and slot array.

Slot Array

Slot array, also known as off-set array, is an array of two-byte values, which is read by the SQL Server in reverse i.e. from the very end of the page. Slot array keep slot points to a byte in the page wherever a record begins. First slot of the record slot array is saved in the last two bytes of the page that points to the page of the first record saved at the page.

Conclusion

One of the essential things that should be kept in mind is that, In SQL Server, the database file (.mdf) store its data as per 8 kb pages and it is important for SQL Server to sustain the sequence of data pages as per their header information. In case of mismatching of information or the data is not stored in sequential order into data pages, there is possibility in the corruption of database or you may face SQL database error like 5172, which is a header corruption error.