Blog 11.02.2025

Mapping out the future of billing unit

Big changes are coming to CSC resource allocations in computing, data and cloud services. Last year, an internal development project to create and implement a new model for managing our resources was conducted. We are switching from an all-encompassing billing unit to more specific units for HPC, HPC GPU, Cloud, and Storage services.

Additionally, there will be changes to quota application processes, and the lifetimes of unused billing units and extra quotas will be limited. Instead of applying for one kind of billing units researchers need to specify their resource needs for HPC, HPC GPU, cloud computing, and storage separately. This means more detailed resource planning and resource applications. Many users have a good idea of what resources and how much of them they need already. If the choice of resources and their quantities present problems, we are happy to help with choosing the right service and assessing the resource needs. In general it’s good to start by testing with a small amount of units and apply for more once the research needs are clear.

In the future, capacity management for the different billing units could look like this:

Interplay of capacity and resource allocation management. Usage tails capacity and typically also allocations, while unused allocations create uncertainty on the availability of capacity, in case dormant service users suddenly activate. Note that CPU, GPU and (different varietes of) storage allocations are not exchangeable for the researcher, while they have been in the traditional billing units.

New billing units will reflect the different kinds of resources and usage

CSC’s services for research need many different kinds of infrastructure. Different research projects have varying requirements: Some need a lot of CPUs or GPUs for a short while, some need more permanent resources, and some need vast amounts of data storage. These requirements can evolve independently over time.

The billing units that CSC has used since 1994 do not differentiate between these needs however. Only ten years ago cloud computing was just a small part of our service offering, but its importance has grown quickly, and the need for storage capacity has grown even quicker. The rule-of-thumb for large resource applications has been to apply for resources to cover the needs for the next six months. Hence some of the allocations have been large and their effect on services has been difficult to assess, leading to capacity shortages and long queues for some resources.

The growing user base creates another reason to streamline processes: Ten years ago CSC had about 6000 users and now there are more than 12 000. In the same time the amount of academic projects has more than tripled from 1300 to 4500.

Another reason for the improved capacity management is the change of CSC’s funding: In the future the ministry funding for research infrastructure is continuous hence the need for better usage data and usage control, which in turn enables better continuous ecosystem and capacity development.

Capacity management the key motivation for billing unit renewal

The main objectives for the change are to improve the capacity management and to unify resource management practices while keeping the model as simple and clear as possible for the end-users. We have examined how resources are managed at other HPC centres and what we can learn from them. As part of the project, quota applications will be centralized to MyCSC, aiming to develop it into a one-stop-shop for all computing project management needs.

In addition to simplified resource management we strive to ensure that researchers have access to the resources they have been granted since the grants will be more specific to services, the granted but unused billing units deprecate if the project doesn’t use them as planned, and with more accurate usage prediction we can invest in the right kind of capacity to match future needs. We are also planning to implement user-friendly visualisations about the capacities in various CSC services.

The billing unit renewal will be implemented in the autumn of 2025 and will possibly be piloted in the spring. The change will affect the services for research: Roihu, Puhti, and Mahti supercomputers, Pouta and Rahti cloud services, SD Connect and SD Desktop services, as well as Allas, Shared fileservice, and Pukki data services – essentially all services using billing units. For any questions, feel free to contact the authors of this blog for more information.

CPU billing units have dominated the overall consumption, but GPU BUs have grown relatively much more and cloud usage has become clearly more significant part of the resources. Note, that for many researchers these resources are not interchangeable and hence it can overload individual resources if the (old generic) allocated BU usage focuses on one of them.

Include publications for resource application directly from Research Hub

CSC has also made it possible for researchers to provide publication and research funding information from Research Hub while filling in the resource application in MyCSC customer portal. This kind of additional information gives CSC resource allocation process more information about the qualifications and impact of the research, and hopefully raises the level of resource allocations. The results of using CSC’s services (publications, positive funding decisions) are an integral part of CSC’s impact assessments. We strive to allocate our resources according to the guidelines from the Ministry of Education and Culture as well as the EU, fitting together the needs of top research, the feedback from our users, and the usage and application data we collect.

Billing unit renewal supports the implementation of updated resource allocation policy

The billing unit renewal described in this blog is closely related to other improvements CSC has done recently. CSC resource allocation policy, a high-level policy to guide the scientific resource allocation, has been updated to meet the needs of a wider service catalog. Other policies and practices are being updated at the same time as well. Requiring a machine actionable data management plan for large storage quota applications in the future is planned. This is to make a more informed decisions based on the plans for the data during and after the research project.

We will communicate the changes and their schedule later. Summary of planned accounting model for services for research: