Tag Archive for 'NameNode'

WANdisco’s March Roundup

Following the recent issuance of our “Distributed computing systems and system components thereof” patent, which cover the fundamentals of active-active replication over a Wide Area Network, we’re excited to announce the filing of three more patents. These patents involve methods, devices and systems that enhance security, reliability, flexibility and efficiency in the field of distributed computing and will have significant benefits for users of our Hadoop Big Data product line.

“Our team continues to break new ground in the field of distributed computing technology,” said David Richards, CEO for WANdisco. “We are proud to have some of the world’s most talented engineers in this field working for us and look forward to the eventual approval of these most recent patent applications. We are particularly excited about their application in our new Big Data product line.”

Our Big Data product line includes Non-Stop NameNode, WANdisco Hadoop Console and WANdisco Distro (WDD.)

This month, we also welcomed Bas Nijjer, who built CollabNet UK from startup to multimillion dollar recurring revenue, to the WANdisco team. Bas Nijjer has a proven track record of increasing customer wins, accelerating revenue and providing customer satisfaction, and he takes on the role of WANdisco Sales Director, EMEA.

“Bas is an excellent addition to our team, with great insight on developing and strengthening sales teams and customer relationships as well as enterprise software,” said David Richards. “His expertise and familiarity with EMEA and his results-oriented attitude will help strengthen the WANdisco team and increase sales and renewals. We are pleased to have him join us.”

If joining the WANdisco team interests you, visit our Careers page for all the latest employment opportunities.

We’ve also posted lots of new content at the WANdisco blog. Users of SmartSVN, our cross-platform graphical Subversion client, can find out how to get even more out of their installation with our ‘Performing a Reverse Merge in SmartSVN’ and ‘Backing Up Your SmartSVN Data’ tutorials. For users running the latest and greatest, 7.5.4 release of SmartSVN, we’ve put together a deep dive into the fixes and new functionality in this release with our ‘What’s New in SmartSVN 7.5.4?’ post. If you haven’t tried SmartSVN yet, you can claim your free trial of this release by visiting http://smartsvn.com/download

We also have a new post from James Creasy, WANdisco’s Senior Director of Product Management, where he takes a closer look at the “WAN” in “WANdisco:”

“We’ve all heard about the globalization of the world economy. Every globally relevant company is now highly dependent on highly available software, and that software needs to be equally global. However, most systems that these companies rely on were architected with a single machine in mind. These machines were accessed over a LAN (local area network) by mostly co-located teams.

All that changed, starting in the 1990’s with widespread adoption of outsourcing. The WAN computing revolution had begun in earnest.”

You can read “What’s in a name, WANdisco?” in full now.

Also at the blog we address the hot topic of ‘Is Subversion Ready for the Enterprise?’ And, if you need more information on the challenges and available solutions for deploying Subversion in an enterprise environment, be sure to sign up for our free-to-attend ‘Scaling Subversion for the Enterprise’ sessions. Taking place a few times a week, these webinars cover limitations and risks related to globally distributed SVN deployments, as well as free resources and live demos to help you overcome them. Take advantage of the opportunity to get answers to your business-specific questions and live demos of enterprise-class SVN products.

WANdisco Files Three New Patents with USPTO

We are pleased to announce the filing of three new patents with the United States Patent and Trademark Office (USPTO) related to distributed computing.

These three innovations involve methods, devices and systems that enhance security, reliability, flexibility and efficiency in the field of distributed computing. The patents are expected to have significant benefits for users of our new Hadoop Big Data product line.

Our team continues to break new ground in the field of distributed computing technology,” said David Richards, CEO for WANdisco. “We are proud to have some of the world’s most talented engineers in this field working for us and look forward to the eventual approval of these most recent patent applications. We are particularly excited about their application in our new Big Data product line.”

Our Big Data product line includes Non-Stop NameNode, which turns the NameNode into an active-active shared-nothing cluster, and the comprehensive wizard-driven management dashboard ‘WANdisco Hadoop Console.’ We also offer a free-to-download, fully-tested and production-ready version of Apache Hadoop 2. Visit the WANdisco Distro (WDD) to learn more.

This news comes after we announced the issuance of our “Distributed computing systems and system components thereof” patent, which covers the fundamentals of active-active replication over a Wide Area Network.

 

Continuous Availability versus High Availability

Wikipedia’s page on Continuous Availability is available here:

http://en.wikipedia.org/wiki/Continuous_availability

A quick perusal tells us that High Availability can be ‘accomplished by providing redundancy or quickly restarting failed components’. This is very different from ‘Continuously Available’ systems that enable continuous operation through planned and unplanned outages of one or more components.

As large global organizations move from using Hadoop for batch storage and retrieval to mission critical real-time applications where the cost of even one minute of downtime is unacceptable, mere high availability will not be enough.

Solutions such as HDFS NameNode High Availability (NameNode HA) that come with Apache Hadoop 2.0 and Hadoop distributions based on it are subject to downtimes of 5 to 15 minutes.  In addition, NameNode HA, is limited to a single data center, and only one NameNode can be active at a time, creating a performance as well as an availability bottleneck. Deployments that incorporate WANdisco Non-Stop Hadoop are not subject to any downtime, regardless of whether a single NameNode server or an entire data center goes offline. There is no need for maintenance windows with Non-Stop Hadoop, since you can simply bring down the NameNode servers one at a time, and perform your maintenance operations.  The remaining active NameNodes continue to support real-time client applications as well as batch jobs.

The business advantages of a Continuously Available, multi-data center aware systems are well known to IT decision makers. Here are some examples that illustrate how both real-time and batch applications can benefit and new use cases can be supported:

  • A Batch Big Data DAG is a chain of applications wherein the output of a preceding job is used as the input to a subsequent job. At companies such as Yahoo, these DAGs take six to eight hours to run, and they are run every day. Fifteen minutes of NameNode downtime may cause one of these jobs to fail. As a result of this single failure, the entire DAG may not run to completion, creating delays that can last many hours.
  • Global clickstream analysis applications that enable businesses to see and respond to customer behavior or detect potentially fraudulent activity in real-time.
  • A web site or service built to use HBase as a backing store will be down if the HDFS underlying HBase goes down when the NameNode fails. This is likely to result in lost revenue and erode customer goodwill.  Non-Stop Hadoop eliminates this risk.
  • Continuous Availability systems such as  WANdisco Non-Stop Hadoop are administered with  fewer staff. This is because failure of one out of five NameNodes is not an emergency event. It can be dealt with by staff during regular business hours. Significant cost savings in staffing can be achieved since Continuously Available systems do not require 24×7 sysadmin staff .  In addition, in a distributed multi-data center environment, Non-Stop Hadoop can be managed from one location.
  • There are no passive or standby servers or data centers that essentially sit idle until disaster strikes.  All servers are active and provide full read and write access to the same data at every location.

See a demo of Non-Stop Hadoop for Cloudera and Non-Stop Hadoop for Hortonworks in action and read what leading industry analysts like Gartner’s Merv Adrian have to say about the need for continuous Hadoop availability.

 

avatar

About Jagane Sundar

WANdisco’s February Roundup

This month, we launched a trio of innovative Hadoop products: the world’s first production-ready distro; a wizard-driven management dashboard; and the first and only 100% uptime solution for Apache Hadoop.

hadoop big data

We started this string of Big Data announcements with WANdisco Distro (WDD) a fully tested, free-to-download version of Apache Hadoop 2. WDD is based on the most recent Hadoop release, includes all the latest fixes and undergoes the same rigorous quality assurance process as our enterprise software solutions.

This release paved the way for our enterprise Hadoop solutions, and we announced the WANdisco Hadoop Console (WHC) shortly after. WHC is a plug-and-play solution that makes it easy for enterprises to deploy, monitor and manage their Hadoop implementations, without the need for expert HBase or HDFS knowledge.

The final product in this month’s Big Data announcements was WANdisco Non-Stop NameNode. Our patented technology makes WANdisco Non-Stop Namenode the first and only 100% uptime solution for Hadoop, and offers a string of benefits for enterprise users:

  • Automatic failover and recovery
  • Automatic continuous hot backup
  • Removes single point of failure
  • Eliminates downtime and data loss
  • Every NameNode server is active and supports simultaneous read and write requests
  • Full support for HBase

To support the needs of the Apache Hadoop community, we’ve also launched a dedicated Hadoop forum. At this forum, users can get advice on their Hadoop installation and connect with fellow users, including WANdisco’s core Apache Hadoop developers Dr. Konstantin V. Shvachko, Dr. Konstantin Boudnik, and Jagane Sundar.

subversion

For Apache Subversion users, we announced the next webinars in our free training series:

  • Subversion Administration – everything you need to administer a Subversion development environment
  • Introduction to SmartSVN – a short introduction to how Subversion works with the SmartSVN graphical client
  • Checkout Command – how to get the most out of the checkout command, and the meaning of the various error messages you may encounter
  • Commit Command – learn more about this command, including diff usage, working with unversioned files and changelists
  • Introduction to Git – everything a new user needs to get started with Git
  • Hook Scripts – how to use hook scripts to automate tasks such as email notifications, backups and access control
  • Advanced Hook Scripts – an advanced look at hook scripts, including using a config file with hook scripts and passing data to hook scripts

We’ve announced an ongoing series of free webinars, which demonstrate how you can overcome these challenges from an administrative, business and IT perspective, and get the most out of deploying Subversion in an enterprise environment. These ‘Scaling Subversion for the Enterprise’ webinars will be conducted by our expert Solution Architect three times a week (Tuesday, Wednesday and Thursday) at 10.00am PST/1.00pm EST, and will cover:

  • The latest technology that can help you overcome the limitations and risks associated with globally distributed deployments
  • Answers to your business-specific questions
  • How to solve critical issues
  • The free resources and offers that can help solve your business challenges

WANdisco Non-Stop NameNode Removes Hadoop’s Single Point of Failure

We’re pleased to announce the release of the WANdisco Non-Stop NameNode, the only 100% uptime solution for Apache Hadoop. Built on our Non-Stop patented technology, Hadoop’s NameNode is no longer a single point of failure, delivering immediate and automatic failover and recovery whenever a server goes offline, without any downtime or data loss.

“This announcement demonstrates our commitment to enterprises looking to deploy Hadoop in their production environments today,” said David Richards, President and CEO of WANdisco. “If the NameNode is unavailable, the Hadoop cluster goes down. With other solutions, a single NameNode server actively supports client requests and complex procedures are required if a failure occurs. The Non-Stop NameNode eliminates those issues and also allows for planned maintenance without downtime. WANdisco provides 100% uptime with unmatched scalability and performance.”

Additional benefits of Non-Stop NameNode include:

  • Every NameNode server is active and supports simultaneous read and write requests.
  • All servers are continuously synchronized.
  • Automatic continuous hot backup.
  • Immediate and automatic recovery after planned or unplanned outages, without the need for administrator intervention.
  • Protection from “split-brain” where the backup server becomes active before the active server is completely offline. This can result in data corruption.
  • Full support for HBase.
  • Works with Apache Hadoop 2.0 and CDH 4.1.

“Hadoop was not originally developed to support real-time, mission critical applications, and thus its inherent single point of failure was not a major issue of concern,” said Jeff Kelly, Big Data Analyst at Wikibon. “But as Hadoop gains mainstream adoption, traditional enterprises rightly are looking to Hadoop to support both batch analytics and mission critical apps. With WANdisco’s unique Non-Stop NameNode approach, enterprises can feel confident that mission critical applications running on Hadoop, and specifically HBase, are not at risk of data loss due to a NameNode failure because, in fact, there is no single NameNode. This is a major step forward for Hadoop.”

You can learn more about the Non-Stop NameNode at the product page, where you can also claim your free trial.

If you’d like to get first-hand experience of the Non-Stop NameNode and are attending the Strata Conference in Santa Clara this week, you can find us at booth 317, where members of the WANdisco team will be doing live demos of Non-Stop NameNode throughout the event.

WANdisco Joins Fusion-io Technology Alliance Program

WANdisco is excited to announce its partnership with Fusion-io. Following the launch of our first Big Data offering, the world’s first production-ready Apache Hadoop 2 distro, we’ve joined Fusion-io’s Technology Alliance Program. This program focuses on working with leaders in strategic market segments to deliver proven solutions, access to resources and expertise to enhance the value of technology offerings.

“With rapid growth in big data demands around the world, customers require proven solutions and expertise that deliver Hadoop availability with no downtime or data loss,” said Tyler Smith, Fusion-io’s Vice President of Alliances. “WANdisco is a valuable addition to our Technology Alliance Program as we work together to fulfill the market demand for innovative and proven big data solutions.”

As mentioned, this partnership news follows the launch of WANdisco Distro (WDD), a fully tested, production-ready version of Apache Hadoop, based on the most recent Hadoop release. WDD lays the foundation for WANdisco’s upcoming enterprise Hadoop solutions, including the WANdisco Hadoop Console, a comprehensive, wizard-driven management dashboard and the Non-Stop NameNode, which combines our patented replication technology with open source Hadoop to deliver optimum performance, scalability and availability on a 24-by-7 basis.

You can find out more about the Technology Alliance announcement by reading the press release, or visiting Fusion-io’s Technology Alliance Program webpage.

WANdisco Launches World’s First Production-Ready Apache Hadoop 2 Distro

hadoop

We’re excited to announce the launch of our WANdisco Distro (WDD) a fully tested, production-ready version of Apache Hadoop 2. WDD is based on the most recent Hadoop release, includes all the latest fixes and undergoes the same rigorous quality assurance process as our enterprise software solutions.

The team behind WDD is led by Dr. Konstantin Boudnik, who is one of the original Hadoop developers, has been an Apache Hadoop committer since 2009 and served as a Hadoop architect with Yahoo! This dedicated team of Apache Hadoop development, QA and support professionals is focused exclusively on delivering the highest quality version of the software.

We are also now offering enterprise-class professional support for organizations deploying Hadoop clusters that utilize WDD. Delivered by our team of open source experts, WANdisco’s professional support for Hadoop includes online service request and case tracking, customer discussion forums, online access to service packs and patches, indemnification coverage, Hadoop cluster health checks, consulting and training and more. You can find out more about the available support options at www.wandisco.com/support/hadoop

We’re particularly excited to make this announcement, as WDD lays the foundation for our enterprise Hadoop solutions that deliver 24-by-7 availability, scalability and performance globally, without any downtime or data loss.

“This is one of a number of key Big Data product announcements WANdisco will be making between now and the upcoming Strata 2013 Big Data conference in Santa Clara, CA, February 26-28. It’s a great time for enterprises requiring a hardened, non-stop Hadoop,” said David Richards, CEO of WANdisco. “Only our patented active-active technology removes the single point of failure inherent in Hadoop and works locally and globally. We are excited to have Dr. Konstantin Boudnik, one of the original developers of Hadoop, leading this rollout.”

You can learn more about WDD at the official press release, or by visiting the Download WANdisco Distro webpage.

WANdisco Teams up with Cloudera

We’re pleased to announce that WANdisco is now an authorized member of the Cloudera Connect Partner Program. This program focuses on accelerating the innovative use of Apache Hadoop for a range of business applications.

“We are pleased to welcome WANdisco into the Cloudera Connect network of valued service and solution providers for Apache Hadoop and look forward to working together to bring the power of Big Data to more enterprises,” said Tim Stevens, Vice President of Business and Corporate Development at Cloudera. “As a trusted partner, we will equip WANdisco with the tools and resources necessary to support, manage and innovate with Apache Hadoop-based solutions.”

As a member of Cloudera Connect, we are proud to add Cloudera’s extensive tools, use case insight and resources to the expertise of our core Hadoop committers.

You can learn more about this program at Cloudera’s website and by reading the official announcement in full.

At WANdisco, we’re working on our Hadoop-based products, including WANdisco Non-Stop NameNode, which will enable each NameNode server to support simultaneous read and write requests, alongside balancing workload across servers for optimum scalability and performance.

You can learn more about Non-Stop NameNode, and our other upcoming Hadoop-based offerings at our Hadoop Big Data Products page.

WANdisco’s January Roundup

Happy new year from WANdisco!

This month we have plenty of news related to our move into the exciting world of Apache Hadoop. Not only did another veteran Hadoop developer join our ever-expanding team of experts, but we announced a partnership with Cloudera, and WANdisco CEO David Richards and Vice President of Big Data Jagane Sundar met with Wikibon’s lead analyst for an in-depth discussion on active-active big data deployments.

WANdisco big data

You may have heard that AltoStor founders and core Apache Hadoop creators, Dr. Konstantin Shvachko and Jagane Sundar joined WANdisco last year. Now we’re excited to announce that another veteran Hadoop developer has joined our Big Data team. Dr Konstantin Boudnik is the founder of Apache BigTop and was a member of the original Hadoop development team. Dr. Boudnik will act as WANdisco’s Director of Big Data Distribution, leading WANdisco’s Big Data team in the rollout of certified Hadoop binaries and graphical user interface. Dr. Boudnik will ensure quality control and stability of the Hadoop open source code.

In building our Big Data team, we’ve been seeking Hadoop visionaries and authorities who demonstrate leadership and originality,” said David Richards, CEO of WANdisco. “Konstantin Boudnik clearly fits that description, and we’re honored that he’s chosen to join our team. He brings great professionalism and distribution expertise to WANdisco.”

Also on the Big Data-front, CEO David Richards, and Vice President of Big Data Jagane Sundar, spoke to Wikibon’s lead analyst about our upcoming solution for active-active big data deployments.

We can take our secret sauce, which is this patented active-active replication algorithm, and apply it to Hadoop to make it bullet-proof for enterprise deployments,” said David Richards. “We have something coming out called the Non-Stop NameNode … that will ensure that Hadoop stays up 100% of the time, guaranteed.”

Watch the ‘WANdisco Hardening Hadoop for the Enterprise’ video in full, or read Wikibon’s Lead Big Data Analyst Jeff Kelly’s post about the upcoming Non-Stop NameNode.

Capping off our Big Data announcements, WANdisco is now an authorized member of the Cloudera Connect Partner Program. This program focuses on accelerating the innovative use of Apache Hadoop for a range of business applications.

We are pleased to welcome WANdisco into the Cloudera Connect network of valued service and solution providers for Apache Hadoop and look forward to working together to bring the power of Big Data to more enterprises,” said Tim Stevens, Vice President of Business and Corporate Development at Cloudera. “As a trusted partner, we will equip WANdisco with the tools and resources necessary to support, manage and innovate with Apache Hadoop-based solutions.”

As a member of Cloudera Connect, we are proud to add Cloudera’s extensive tools, use case insight and resources to the expertise of our core Hadoop committers.

You can learn more about this program at Cloudera’s website and by reading the official announcement in full.

apache subversion logo

On the Subversion side of things, the SVN community announced their first release of 2013, with an update to the Subversion 1.6 series.

Apache Subversion 1.6.20 includes some useful fixes for 1.6.x users:

  • Vary: header added to GET responses
  • Fix fs_fs to cleanup after failed rep transmission.
  • A fix for an assert with SVNAutoVersioning in mod_dav_svn

Full details on Apache Subversion 1.6.20 can be found in the Changes file. As always, the latest, certified binaries can be downloaded for free from our website, along with the latest release of the Subversion 1.7 series.

How many developers can a single Apache Subversion server support? In his recent blog post, James Creasy discussed how DConE replication technology can support Subversion deployments of 20,000 or more developers.

“While impressive, DConE is not magic,” writes James. “What DConE delivers is a completely fault tolerant, mathematically ideal coordination engine for performing WAN connected replication.”

In another new DConeE post, James explains where DConE fits into the ‘software engineering vs. computer science’ debate, and warns “in the world of distributed computing, you’d better come armed with deep knowledge of the science.”

Finally, WANdisco China, a Wholly Foreign Owned Enterprise was announced this month, following WANdisco’s first deal in China with major telecommunications equipment company Huawei. From this new office we’ll be providing sales, training, consulting and 24/7 customer support for WANdisco software solutions sold in China, and are excited to be expanding our activities within this region.

We view China as an emerging and high growth market for WANdisco,” said David Richards. “It was a natural progression to establish our Chengdu office as a WFOE and ramp up staff there as so many companies have operations in the country. We are excited about this announcement and look forward to the growth opportunities this brings.”

To keep up with all the latest WANdisco news, be sure to follow us on Twitter.

 

WANdisco’s December Roundup

2012 has been an amazing year for WANdisco, but we still had a few more announcements for you this month, including news that we are extending our suite of service and support offerings to include the Git distributed version control system.

“Expanding our support offering to include Git is an obvious step to enable you to deploy and support the trending as well as the leading SCM tools,” said James Creasy, WANdisco’s Senior Director of Product Management in his ‘Supporting Git to Support You’ blog.

Our Git support includes:

  • Guaranteed response times
  • Availability 24 hours a day, 7 days a week
  • Contact via email or toll-free telephone

Git support is available immediately, please contact sales@wandisco.com for more information.

This news comes hot on the heels of our Big Data and Apache Hadoop announcements last month. WANdisco CEO, David Richards, and core creators of Apache Hadoop Dr. Konstantin Shvachko and Jagane Sundar recently conducted a webinar that covered how WANdisco sees the future of big data, following our acquisition of AltoStor.

This 30 minute webinar discussed:

  • The cross-industry growth of Hadoop in the enterprise.
  • How Hadoop’s limitations, including HDFS’s single-point of failure, are impacting the productivity of the enterprise.
  • How WANdisco’s replication technology will alleviate these issues by adding high-availability, data replication and data security to Hadoop.

If you missed out on the webinar, you can still find out all about WANdisco, Hadoop and Big Data by checking out the webinar slides on SlideShare.

The Subversion community also found the time for one more release before the holiday season. Subversion 1.7.8 features plenty of fixes and enhancements, including:

  • Adding missing attributes to “svn log -v –xml” output
  • Fixing a hang that could occur during error processing
  • Fixing incorrect status returned by 1.6 API
  • Adding Vary: header to GET responses to improve cacheability
  • Subversion 1.7.8 ignores file externals with mergeinfo when merging

A full list of everything that’s new in Subversion 1.7.8 is available at the Changes file. Free binaries of Subversion 1.7.8 are available to download through the WANdisco website. Users of SmartSVN, the popular cross-platform client for Subversion can also grab an update: SmartSVN 7.5.3 features plenty of improvements and bug fixes, including:

  • Refresh option to ask for master password, if required
  • Support launching on Solaris
  • Fix for an internal error that could occur after removing Tag
  • Special characters (e.g ‘:’) no longer cause problems in URLs

More information on the latest changes, is available at the SmartSVN changelog. If you haven’t tried SmartSVN yet, remember you can claim your 30 day free trial of SmartSVN Professional by visiting www.smartsvn.com/download

There’s been plenty of new content at the blog this month, including the first blog from Hadoop core creator Jagane Sundar, WANdisco’s new Vice President of Engineering of Big Data.

In his ‘Design of the Hadoop HDFS NameNode: Part 1 – Request processing’ post, Jagane demonstrates how a client RPC request to the Hadoop HDFS NameNode flows through the NameNode.

hadoop namenode

When you think of “the cloud”, what comes to mind? In his first WANdisco blog, Director of Product Management James Creasy takes a fresh look at one of IT’s biggest buzzwords. He argues that most of the applications used by enterprises were not originally architected for cloud infrastructures, and looks at how this problem could be overcome by “putting the cloud into a virtual eyedropper.” In his second blog, ‘Planned Downtime Is Still Downtime’ James argues that planned outages of critical applications aren’t inevitable:

Through the 20th century and into the 21st we’ve gritted our teeth against this inescapable cost. We’ve built massive failover servers, concocted elaborate master/slave replication schemes, and built businesses around High Availability and Disaster Recovery scenarios (HADR). We thought we were doing the best we can.

And we were, until recently.”

You can read the ‘Planned Downtime is Still Downtime’ post in full at the WANdisco blog.

We also had some new team photos taken by our friend and neighbour at our Electric Works offices, Matt Lollar. We even managed to get some shots outside in the Sheffield sunshine.

wandisco team

Finally, to celebrate the holiday season, we had a little ‘Decorate Your Desk’ competition in the Sheffield office. We even had a roaring log fire!

christmas fire

We have plenty of exciting announcements planned for 2013, but for now we’d just like to thank everyone who has used our products, joined us for a webinar, eTraining or enterprise training session, picked us for your support needs, or provided the crucial feedback we need to make our products and services even better. And, of course, we’d like to wish you a very happy holidays from the WANdisco Team.