Blog Listing

Thursday, 09 December 2021 14:49

Datacentre Backup Diesel Generators Winter Maintenance

Written by

 

Datacenter Backup Diesel Generators Winter Maintenance

 

 

Backup generators are vital for proper datacenter operation. Regular maintenance procedures are mandatory to ensure that your units will start smoothly and supply necessary power to your facilities.

 

The probability for power outages is higher during the winter season, due to possible overloading of power grid in very cold days or after snow/wind storms and other natural disasters. From other hand, internal combustion engines need special care at low ambient temperatures, especially during staritng and warming up. Therefore in TSBG hosting ltd. we plan diesel generators maintenance just before the winter. October and November are the months, best suited for that.  

 

Our team would like to share what you have to pay attention and take care to ensure that you are ready for the winter.

 

OIL CHANGE

 

Oil filter change is regular maintenance procedure for any internal combustion engine. As, in most cases, generators are driven by diesel engines, you need to replace oil on regular intervals, as specified by the engine manufacturer. Usually this interval is 200-300 hours of operation. All our datacenters are designed with redundant power supply system and grid power outages are rare. Therefore the generators are not operating more than 50 hours per year. Nevertheless the engine working hours are less than recommended oil change interval, the oil ages, even without engine operation. Changing oil at least once per year is recommended. Changing on longer intervals is acceptable only if a probe is taken and send to the lab for evaluation. In any case, the price for changing oil is relatively low, compared to overall datacenter maintenance cost and maintaining good oil parameters by regular oil change will ensure long and trouble free operation of the engine and your generator will remain operational for many years.

 

When replacing oil don’t forget to change also oil filter and fuel filter as this is recommended good practice. The air filter needs to be inspect and replaced if dirty or damaged. When the engine is not running the air filter is not collecting dust and it is acceptable to replace it at longer intervals. Please have in mind that the air filter condition depends not only on engine running hours but also on the environment. If it is too dusty or the filter is overheated or there was leakage of water, oil or coolant over the filter, it may become compromised and prevent normal air flow or lose its filter capabilities.

 

If the air filter is blocked and can not provide necessary air flow the engine output power will be reduced and eventually the engine may not start. If the filter can not provide filtering of the engine incoming air, that may result in engine damage.

 

Before oil and filter replacement it is good to run the generator for 15 minutes. That will warm the oil and make it easier and faster to drain it from the engine. Bigger engines needs more oil and draining the oil sometimes take long time, especially when it is very cold.

 

Shut down the generator and block automatic start. Consult with generator manual. The best way is to block generator controls and disconnect battery. Accidental generator start, while replacing oil may cause injuries and damage the engine.

 

Please pay attention to the environment. Always use oil drain pan with sufficient capacity. Store used oil in a plastic or steel containers and return it to local recycling center. Avoid any leakage or spill.

 

 

In case of oil spill do the following (be prepared with necessary materials before staring oil change):

 

  • Sprinkle sawdust, clay kitty litter (not the clumping kind), coconut husks, or a commercial oil-absorbing product on the stain. If you have none of these products on hand, set paper towels to soak up any wet oil.
  • Allow the absorbent materials to work on the oil for 24 to 48 hours. Then, sweep up the debris with a broom and dustpan.
  • Repeat with fresh absorbent materials as needed to remove any remaining oil.
  • Dispose off debris properly-consult local recycling center.
  • Clean the area with water and liquid soap.

 

Protect your eyes, skin and clothes.

 

Attention! The hot engine oil can burn your eyes and skin. Don’t attempt oil replacement if the generator was operational for long time. Wait enough for the oil to cool down to 35 degrees C (95 degrees F).

 

After the oil is drained, close the drain opening and dismount oil filter. Please note that there is small amount of oil remained in the filter. Follow manufacturers manual how to remove and replace oil filter. In most cases you will need a special tool.

 

 

Before fitting the new oil filter don’t forget to fill it with fresh oil. That will make engine start easier and avoid engine running without oil during filling of oil system.

 

Dispose old oil filter properly. Consult local oil recycling center.

 

Fill the engine with a fresh oil with grade and type as specified by manufacturer. Always use high quality oil. Quality of the engine oil is critical for engine life, cold start capabilities and even fuel consumption.

 

If manufacturer recommends oil with different viscosity, select the one that best match your local environment conditions during the period  between oil changes. An example chart is picture below, but always use manufacturer maintenance manual.

 


 

If the generator is equipped with a canopy filling the engine oil may be difficult.

 

After oil filter replacement, proceed to fuel filter replacement. Usually fuel filter is fitted by a bolt and no special tool is required. Consult engine maintenance manual. Please note that the old filter has some amount of fuel in it. Before fitting the new filter, fill it with fuel. That will ease engine start. If you need to detach fuel hoses, depending on the clamps, you may need a special clamp tool like the one shown on the picture

 

 

 

 

Attention! The engine fuels are highly flammable. Do not smoke and use of fire and heating tools when working with fuels. Always prevent your eyes, skin and clothes. Avoid fuel spill.

 

 

 

After replacing fuel filter inspect and if necessary clean or replace air filter. Small amounts of dirt can be spraying with compressed air.

 

Fuel hoses and fuel system inspection

 

The rubber or plastic which fuel hoses are made of ages with time. Carefully inspect all hoses from fuel tank to fuel pump, fuel filter and engine assembly. Sometimes small cracks appear in the fuel supply hoses. The cracks let air enter in the fuel hoses thus preventing necessary fuel flow. The result is that the engine operates normal when iddle, but when the generator is loaded the fuel flow is not enough to supply necessary amount of fuel to the engine and the generator starts behaving as it is overloaded with uneven operation and drop in supply voltage. In fact the generator is unusable and diagnosing the problem is not easy and often mistaken.

 

If you find any problem with the fuel hose, replace all of them. The cost is minor and may save you a lot of troubles. Always use high quality hoses. Please note that sometimes fuel hoses are from different manufacturers for engine and generator and one type can look OK but another type may be compromised.

 

 Don’t forget to inspect and replace additional fuel filter between fuel tank and the engine, if such additional filter is installed.

 

CHECK COOLANT

Engine coolant is a mixture of ethylene-glycol and water. Mixture ratio determines freezing temperature of the coolant. It is a must to ensure that the freezing temperature of the coolant is lower than the lowest expected air temperature during the winter. Winter temperatures depend on geographic location but may vary a lot from year to year. Usually generator assembly contain coolant heater to keep the engine warm and ease quick starting. Freezing of the coolant will surely cause severe damage of the engine. Therefore to be on the safe side, in any case keep freezing point of the coolant lower than -20 degrees C.

 

You can measure the freezing temperature with a simple and very useful tool, which I recommend to be added to maintenance toolbox of every site, having backup diesel generators or any other device, driven by internal combustion engine with liquid cooling system.

 

        

 

Check freezing temperature of the coolant every autumn, before winter. During the test, check coolant color and transparency. If the coolant is dirty or unclear, drain the coolant and flush the system several times. Always use distilled water.  The color of healthy engine coolant is green (for ethylene glycol) or orange (for Dexcool). A rusty color indicates that the rust inhibitor in the coolant has broken down and it can no longer control rust and scale buildup. A milky color indicates the presence of oil in the system. Orange coolant normally last longer than the green one.

 

Fill new coolant. After refilling, let the engine operate for 20 minutes and check freezing temperature again as some water, remained in the system may increase freezing temperature, above acceptable limits.

 

Record the last coolant replacement date and plan coolant replacement every 3 years for blue and 5 years for red coolant. Don’t mix different types of antifreeze.

 

CHECK BATTERY

 

Engine battery is required to provide necessary current to start the engine. If not maintained properly or if the charging system is faulty, the battery may be damaged irreversibly. Even well maintained and properly charged, the battery capacity will reduce substantially after 5 years of operation and needs to be replaced. During the winter season, when cold, the diesel engine start is more difficult, due to the nature of ignition process. At the same time battery capacity is reduced and can’t drive necessary current to the engine starter motor, resulting in lower revolutions. In combination both phenomenon.  

 

The battery health is vital for proper operation of the generator. It doesn’t matter how well your generator assembly is maintained if you have a faulty battery the engine will not start when expected.

 

First – check electrolyte level if you don’t have maintenance free battery. If necessary fill with distilled water.

 

Measuring battery voltage can indicate only the level of charging but can not determine battery capacity. You need to test the battery under load to evaluate its state. The best way to perform this is withspecial battery tester tool as shown below:

 

 

The tool has powerful resistor inside and can measure battery voltage drop when high current applied this tool is not expensive and it is valuable measuring device for every battery plant. For more advance measurements it is recommended to use electronic device.

 

If the measurement shows that the battery is OK inspect the electrodes. Clean and apply thin layer of vaseline, which will prevent corrosion. Check also battery compartment for electrolyte spill and rust. If necessary clean, prime and paint.

 

Replace battery if the measurement shows weak battery or if it is more than 5 years old. Install maintenance free battery with the same capacity. If the battery is older than 3 year make battery test every month or two. This will save you a lot of troubles.

 

 

CHECK COOLANT HEATER

Don;t forget to check and if necessary adjust coolant heater. Fenerator assemblies are equipped with coolant heater which keeps the engine warm and ease starting. During the winter it is very important to have this operational. Raplace faulty heater if necessary.

FILL FUEL TANK

Fill fuel tank to the top. Having enough fuel will guarantee operation of the unit for as long as possible. Revise your contracts for fuel delivery on the site. Big generators require a lot of fuel.

CHECK BELTS

 Check all belts for craks. Check tension. If necessary replace old belts.

TEST THE GENERATOR UNDER LOAD

 At the end start the generator under load for at least 1 hour. During that period check the engine temperature, measure supplied voltage, check for alarms, strange noice and other abnormalities.

 

 

 

 

 

 

 

 

 

Monday, 27 September 2021 22:52

Temperature Monitoring in the Datacentre

Written by

 

Temperature monitoring in the datacenter

 

Monitoring of the environment parameters in a datacenter is very important for normal operation. Moreover, having clear picture of the current values, trends and deviations will ensure system uptime, optimize performance and reduce operation costs.

 

Properly setting alarm triggers will send early alarm notices and give operations staff more time to eliminate faults and manage repairs.

 

 There are no doubts that monitoring system must be adapted to the specific design and purpose of the infrastructure, equipment sensitivity, air conditioning system redundancy etc.

 

E.g. monitoring of high power density server racks in hot/cold aisle configuration is different from equipment container, exposed on the elements at a remote location.

 

Last, but not least, monitoring system design and operation falls outside the core tasks of network engineers and administrators, it often requires different and specific skills and malfunctioning or improper functioning often has no direct impact on operations - especially in early stage. It is therefore often underestimated and left “on hold” and “to solve it later” until disaster strikes.

 

In order to properly design, commission, operate and maintain our critical infrastructure monitoring system we need to identify the key parameters and characteristics such system must have.

 

In this article I will try to summarize all aspects we need to consider when solving this task-both general and specific ones. Your project may differ, but I will try to give you a solution, flexible enough to answer most needs, scopes and sizes.

 

What are the needs

 

Let’s see what problems we must address with our monitoring system.

 

-to be easy to install in diverse environments;

 

-to not disturb operation of other systems;

 

-to operate on 24/7/365 basis;

 

-to be in a redundant configuration;

 

-to keep history data to a predefined period of time;

 

-to be scalable and adjustable;

 

-to generate and clear alarm alerts via different media and at predefined thresholds with predefined severity level;

 

-to be easy to integrate it in other management and monitoring systems;

 

-to stay within reasonable budget limits;

 

-to occupy small space in the rack and in the datacenter and to use small power;

 

-to not interfere with other systems;

 

-to be able to express the data in different formats for preselected periods of time and to be easy to switch between general view and detailed view;

 

-to be able to backup configuration and stored data

 

-to easily add, delete and modify configuration, following current changes and demands;

 

And last, but not least – to comply with the standards

 

 

 

Talking about standards, it is important to mention ASHRAE 90.4, finalized and went into effect in 2016 and recently replaced by ASHRAE 90.4-2019.

 

ASHRAE 90.4 standard establishes the minimum energy efficiency requirements of data centers for design, construction, operation and maintenance and utilization of on-site or off-site renewable energy resources. The standard also stipulates means for temperature monitoring, requiring minimum 3 sensors at the inlet and outlet of every rack-one at the top, middle and bottom section. That means that monitoring system must maintain minimum six sensors per equipment rack.

 

The solution

 

There are many “turn-key” and ready to use monitoring solutions on the market. You can find information by searching the Internet or talking to your datacenter construction/maintenance contractor. In my opinion almost all of them are either not answering all problems, as listed above, or are too expensive, or require additional work/upgrade to be done in order to achieve full functionality. E.g. there are nice monitoring systems delivered with WEB interface only. They have SNMP interface but need extra work to integrate them in operator’s existing SNMP monitoring system or develop a complete new one, dedicated to environment monitoring only.

 

Below I would like to show how you can create and integrate excellent environment monitoring system within very small budget.

 

The components

 

We will need good, reliable and cheap sensors, easily addressable, with compact size and simple and easy deployable cabling.

 

We will need a server, capable of retrieving information, storing, visualizing and analyzing it and triggering alarms.

 

We will need converter, capable of converting sensor’s data into format, readable by the server.

 

Sensors

For this project we have selected addressable 1-wire sensors. There are many options here, but our team likes very much DS18B20 sensor as shown on the picture below. It is water and dust proof with index of protection IP 67. The sensor has stainless steel protective cap with diameter just 6mm and length 50mm and comes with 1 m pre-terminated cable