Revolutionize Data Center Management with NETCONF: The Future of Automation

Exploration of Data Center Automated Operation and Maintenance Technology: NETCONF

Time: September 30th, 2024

Data center switches realize automated operation and maintenance (O&M). Zero configuration online technology (ZAM) only completes the fixed configuration known in advance, mainly some basic intercommunication and login-related configuration. Future services are changing, so specific service-related configurations need to be configured or updated according to particular needs. NETCONF (Network Configuration Protocols) protocol provides a solution for automated service configuration deployment.

So how did NETCONF come about? Why did NETCONF need to be developed? What can NETCONF do? What is its development trend? This article will introduce and discuss data center operation and maintenance automation technologies in response to the above questions.

Problems with SNMP

When talking about NETCONF, SNMP cannot be avoided. SNMP (Simple Network Management Protocol) was first developed by IETF (Internet Engineering Task Force) in the late 1980s. Since its birth, SNMP has been used to monitor (such as alarms and performance management) network devices. Most professional network devices have SNMP agents. These agents are activated and configured to communicate with SNMP management NMS (Network Management System).

A complete SNMP system includes the management information base (MIB), structure of management information (SMI), and SNMP message protocol.

1. MIB

MIB, a Management Information Base, summarizes and stores the unique correspondence between resources and OIDs. NMS finds the corresponding OID in MIB according to the queried resources and sends the read OID information to the network device through an SNMP query message. The network device queries the corresponding resource information according to OID, and finally encapsulates it into an SNMP response message and sends it to NMS.

As shown in Figure 1, the OID corresponding to the query resource iso.org.dod.internet.mgmt.mib.ip.ipInReceives is 1.3.6.1.2.1.4.3.

Figure 1: MIB tree hierarchy

2. SMI

SMI (Structure of Management Information), is a language used in SNMP to define specific data in managed network entities (i.e., network devices). It defines data types, object models, and rules for writing and modifying management information.

3. SNMP Message

SNMP specifies five types of protocol data units (PDUs) (SNMP messages) for information exchange between management processes and agents.

As shown in Figure 2:

● get-request, used to retrieve one or more parameter values from the proxy process;

● get-next-request, used to retrieve the next parameter value following the current parameter value from the proxy process;

● set-request, used to set one or more parameter values of the proxy process;

● get-response, used for one or more parameter values returned by the agent to the management process;

● Trap: A message actively sent by the agent process to notify the management process that certain events have occurred.

Figure 2: Five types of SNMP message operations

From the design of the SNMP protocol, we can see that although it has some configuration functions, SNMP itself is not a configuration-oriented protocol, nor is it suitable for developing client applications for configuration. A large number of operation and maintenance practices have also proved that SNMP is more of a tool for monitoring the status of network devices, and at the configuration management level, SNMP cannot yet become a mature tool.

Even at the network device monitoring level, SNMP still has other problems, such as:

● Poor performance and low efficiency;

● Based on UDP protocol transmission, the reliability is poor and can only rely on its mechanism to ensure reliability, which affects performance;

● Private MIB makes it difficult to uniformly manage devices from multiple manufacturers.

It is precisely because of the various deficiencies and defects of SNMP that NETCONF was created.

The origin of NETCONF

In 2001 and 2002, the IAB ( Internet Architecture Board) organized several special working meetings on network management, bringing together network managers and protocol developers to discuss the problems of the mainstream network management protocols ( SNMP, CLI, etc.) simultaneously. The meeting results eventually formed RFC 3535. In this document, 14 requirements were proposed to address the problems in network management at the time, the most critical of which are:

● Ease of use;

● Clearly distinguish between configuration data, data describing operational status, and statistical data;

● Configuration management for services and networks, rather than individual devices;

● Import and export of configuration data are independent of original personnel operations;

● Text-based configuration;

● Standardized data models, etc.

In response to the requirements listed in RFC 3535, the NETCONF working group was established in 2003, and the design of NETCONF follows RFC 3535. In 2006, the NETCONF core RFC 4741 was released, and in 2011, the updated version of RFC 6241 was released (repealing RFC 4741).

Introduction to NETCONF

NETCONF protocol, as the name implies, is a standard protocol for installing, editing, and deleting network device configurations. As can be seen from the origin of NETCONF above, one of the main problems NETCONF solves is the configuration management, distribution, and modification of network devices. How does NETCONF manage network device configurations? Let's first look at the architecture design of NETCONF.

1. NETCONF Architecture

NETCONF uses a client/server structure and is connection-oriented. Protocol messages are in XML format.

Figure 3: NETCONF protocol architecture

As shown in Figure 3, the NETCONF protocol is divided into four layers, namely, the security transport layer, the message layer, the operation layer, and the content layer.

1. Transport Layer Security

NETCONF is that it stipulates that its transport layer must use secure encryption protocols, such as SSH and TLS. Compared with other plain text transmission protocols, NETCONF guarantees data security at the protocol level. Since the NETCONF protocol stipulates that SSH must be supported, SSH is currently the most widely used transport layer protocol for NETCONF.

2. Message Layer

The message layer provides a simple, transport-independent RPC and notification encapsulation mechanism.

The client uses the <rpc> element to encapsulate the operation request information and sends the request to the server through a secure, connection-oriented session. The server uses the <rpc-reply> element to encapsulate the response information of the RPC request (i.e. the content of the operation layer and content layer) and then sends this response information to the requester. In addition, the server can use the <notification> element to notify the client of events.

3. Operational Layer

The operation layer is the specific operation content carried on the message layer. These specific operations can only be carried out on <rpc> and <rpc-reply> messages. <notification> messages have no operation layer content.

NETCONF protocol specifies nine simple RPC operations, as shown in Table 1:

Basic Operation	Explanation
<get>	Gets configuration data or status data.
<get-config>	Get configuration data.
<edit-config>	Modify Configuration Data (Including merge, replace, create, delete, etc.).
<copy-config>	Replace the target database with a complete database.
<delete-config>	If the database is deleted, the <running/> library cannot be deleted.
<lock>	Lock the device configuration database and grant exclusive database modification permissions, To avoid the occurrence of multi-user operations.
<unlock>	If cancel the device configuration database lock, Only this user's lock can be cancelled.
<close-session>	Request to terminate the NETCONF session normally.
<kill-session>	Forcefully terminating NETCONF session requires administrator privileges.

▲Table 1: Basic RPC operations

In addition to the above 9 basic operations, NETCONF also supports user-defined RPC operations, which are not described in detail here.

4. Content Layer

The content layer is mainly used to describe the configuration data and notification data involved in network management. However, NETCONF does not have a standardized definition of the data structure model of the content layer. So how to describe the relevant data of the content layer? In 2006, when the first version of NETCONF RFC 4741 was released, a data modeling language YANG (RFC 6020) specially prepared for NETCONF was also released. The goal is to model the NETCONF data model and operation, covering the operation layer and content layer of the NETCONF protocol.

The following figure shows the contents of each layer in a NETCONF RPC message.

Figure 4: Example of NETCONF RPC message

2. NECONF Operation

NETCONF implements device configuration management under this architecture.

Let's take a look at the NETCONF interaction process:

Figure 5: NETCONF interaction process diagram

After the client and server establish a NETCONF session, both ends first notify each other of their NETCONF support capabilities, i.e., Capability, through <hello> messages. After synchronization is completed, the client initiates an RPC operation, and the server encapsulates the response message into <rpc-reply> and replies to the client. For device configuration, the client initiates RPC configuration-related operations.

Among the nine basic RPC operations defined by NETCONF, four are directly related to configuration, including:

● <get-config>;

● <edit-config>;

● <copy-config>;

● <delete-config>.

Two remaining items are related to configuration permissions, two session closing items, and one status data query item. This also shows that NETCONF attaches great importance to configuration management.

But these still don't seem to show how NETCONF is better than SNMP? How to solve the problems existing in SNMP?

Here we have to introduce another important component of NETCONF, YANG.

YANG

YANG (Yet another Next Generation), from the YANG standard document RFC 6020, we can see its positioning: A Data Modeling Language for the Network Configuration Protocol (NETCONF), the data modeling language for NETCONF.

1. Why do we need YANG?

NETCONF needs to operate on the configuration and state of the device, such as editing the configuration and obtaining the state, but the content layer of NETCONF does not define a standardized data model. In addition to defining 9 basic RPC operations, the operation layer also allows custom RPCs. Custom RPCs and specific operation contents need to be modeled, and YANG is used to complete this modeling.

YANG uses modules and sub-modules for data modeling. A YANG module, as shown in Figure 6, defines data with a vertical structure that can be used for NETCONF-based operations, including descriptions of configuration and status data, and RPC operations. It enables complete data descriptions between NETCONF clients and servers.

Figure 6: Example of YANG module

2. How to use YANG

YANG has a tree structure. Each node has a name, a value, or some child nodes. The YANG file provides a clear description of these nodes and the interactions between them, that is, the data model path.

NETCONF by network devices registers the YANG data model path on the device so that the device can obtain data or write configuration according to the data model path of the YANG file.

Then, operation and maintenance personnel only need to obtain the YANG model from the equipment manufacturer, fill in the relevant configuration or data parameters according to the requirements of the YANG model, and then initiate relevant RPC operations through NETCONF to complete the configuration or data reading tasks.

Through YANG and NETCONF, network configuration and management based on programmable technology are realized, providing a foundation for automated operation and maintenance.

3. Comparison between NETCONF and SNMP

After introducing NETCONF, let's look back at the simple comparison between NETCONF and SNMP:

	SNMP	NETCONF
Data models	Defined in MIBs	Defined in YANG
Data modeling Language	SMI	YANG
Management Operations	SNMP	NETCONF
RPC Protocol	BER	XML
Transport Stack	UDP	SSH TCP

▲Table 2: Comparison of SNMP and NETCONF

Compared with SNMP, NETCONF has the following advantages:

● Easy to use, the YANG model is easy to learn, readable, and can be directly mapped to XML;

● Clearly distinguish configuration data and status data. Different data are stored in different databases and clearly distinguished from each other;

● Can manage all devices in the entire network in a unified manner, rather than managing each device individually;

● Automated configuration distribution and data collection to reduce the chance of manual errors;

● Standardized data models (modeling languages);

● Good scalability. In addition to standard operations, you can also customize operations to achieve unique management functions;

● Based on SSH protocol, it provides configuration locking and other mechanisms, with high data security.

4. NETCONF Issues

Due to its good usability and scalability, the NETCONF protocol has been increasingly widely used with the gradual implementation of SDN technology. All equipment from mainstream manufacturers can support the NETCONF function.

But does this perfectly solve the problems faced by network management and operation? The answer is no. The reasons are:

● Equipment from various manufacturers that support NETCONF uses the private YANG model. Devices from different manufacturers cannot recognize YANG files from other manufacturers;

● Users of the NETCONF operation and maintenance platform need to learn and be familiar with the NETCONF documents and YANG files of various vendors;

● The overall standardization process of the NETCONF YANG Model is very slow.

In order to solve these practical problems, equipment users in the industry have introduced another standardized solution.

OpenConfig

In order to shield the differences between devices of different manufacturers when using NETCONF, the OpenConfig working group was established by Google, AT&T, British Telecom, Facebook, Apple, Microsoft, etc., hoping to create an open model that is independent of device manufacturers for network configuration and policies. Currently, domestic companies such as Tencent, Baidu, and Alibaba have also joined the OpenConfig working group.

1. OpenConfig Standardization

OpenConfig is to compile a set of neutral common data models using the YANG language based on the operational requirements of actual cases, to ultimately achieve network automation operation and maintenance based on programmable technology.

OpenConfig has the following main features:

● Use the NETCONF protocol framework;

● Use YANG as modeling language;

● Do not modify the underlying data transmission, only focus on upper-level data expression and data modeling.

In other words, OpenConfig is more of a modeling standard.

OpenConfig itself is developed based on the standardized NETCONF protocol framework and modeling language YANG and is standardized in the YANG Model dimension, which is equivalent to standardizing the YANG model. All equipment from various manufacturers must be able to recognize the YANG files of the OpenConfig YANG Model. In this way, equipment from different manufacturers can be managed through the YANG files of the unified model, and the only difference is the parameters filled in the YANG files.

In layman's terms, OC (OpenConfig) YANG hopes to create a public YANG model, as shown in Figure 7.

Figure 7: OC YANG model

2. OpenConfig Ecosystem

Currently, OpenConfig is not an official standard organization or even a formal organization, and many members are invited by email. It is advocated by Google, and Microsoft, AT&T, British Telecom, etc. are all its supporters. BAT in China has joined as well.

▲Figure 8: OpenConfig ecosystem

OpenConfig has not yet released a standard RFC, but it has become the next development trend of NETCONF. Major network equipment manufacturers have begun the development of OpenConfig.

At present, due to its advantages in network management, configuration distribution, and openness, NETCONF has become the mainstream protocol for realizing data center operation and maintenance automation. At the same time, major network equipment manufacturers have also added NETCONF to the function support list. Although there are still some problems in actual application due to the problem of private YANG, OpenConfig has provided a clear and feasible way to solve these problems. With the gradual implementation of OpenConfig, data center automated operation and maintenance technology will usher in a new wave.

Ruijie Networks' data center switch products, including the RG-N18000-X series core switches, RG-S6510 series 25G TOR, and RG-S6220-H series 10G TOR, all support NETCONF and OpenConfig YANG. The OpenConfig YANG models released in the first half of 2018 have also been fully adapted and are currently being tested with domestic public cloud providers.

Summary

Data center switches are increasingly leveraging automated operation and maintenance technologies to enhance efficiency. Zero Configuration Online Technology (ZAM) simplifies initial setup by handling basic configurations like intercommunication and login, but it cannot adapt to the evolving demands of future services. To address this limitation, the Network Configuration Protocol (NETCONF) was developed, providing a robust solution for automated service configuration deployment. NETCONF enables dynamic updates and configurations tailored to specific service requirements, which is a significant advancement over traditional methods. Its development addresses the shortcomings of Simple Network Management Protocol (SNMP), particularly in terms of flexibility and scalability, positioning NETCONF as a key player in the automation landscape of data centers. The trend towards increased automation and real-time adaptability suggests that NETCONF will continue to evolve, becoming integral to future data center operations.

Technical Feast | A Brief Analysis of MMU Waterline Settings in RDMA Network

The Rise of Multi-Gigabit Switches: A Deep Dive

Exploration of Data Center Automated Operation and Maintenance Technology: NETCONF