Search Results for "parallel" - Page 3
Sort By:
Showing 209 open source projects for "parallel"
View related business solutions-
Earn up to 15% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
Earn up to 15% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
OptScale
FinOps and MLOps platform to run ML/AI and regular cloud workloads
Run ML/AI or any type of workload with optimal performance and infrastructure cost. OptScale allows ML teams to multiply the number of ML/AI experiments running in parallel while efficiently managing and minimizing costs associated with cloud and infrastructure resources. OptScale MLOps capabilities include ML model leaderboards, performance bottleneck identification and optimization, bulk run of ML/AI experiments, experiment tracking, and more. The solution enables ML/AI engineers to run automated experiments based on datasets and hyperparameter conditions within the defined infrastructure budget. ... -
2
nbdev
Create delightful software with Jupyter Notebooks
nbdev is a notebook-driven development platform (by fast.ai/AnswerDotAI) enabling you to write code, tests, documentation, and deploy software, all from Jupyter Notebooks. It provides a unified literate programming workflow where you can tag notebook cells for export to Python modules, auto-generate documentation via Quarto (and host it on GitHub Pages), run tests embedded in notebooks, manage clean notebooks with Git-friendly metadata hooks, and seamlessly publish packages to PyPI/conda,... -
3
PyOpenCL
OpenCL integration for Python, plus shiny features
PyOpenCL is a Python wrapper for the OpenCL framework, providing seamless access to parallel computing on CPUs, GPUs, and other accelerators. It enables developers to harness the full power of heterogeneous computing directly from Python, combining Python’s ease of use with the performance benefits of OpenCL. PyOpenCL also includes convenient features for managing memory, compiling kernels, and interfacing with NumPy, making it a preferred choice in scientific computing, data analysis, and machine learning workflows that demand acceleration. -
4
Llama Recipes
Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT method
The 'llama-recipes' repository is a companion to the Meta Llama models. We support the latest version, Llama 3.1, in this repository. The goal is to provide a scalable library for fine-tuning Meta Llama models, along with some example scripts and notebooks to quickly get started with using the models in a variety of use-cases, including fine-tuning for domain adaptation and building LLM-based applications with Llama and other tools in the LLM ecosystem. The examples here showcase how to run... -
Enterprise-Class Managed File Transfer.Diplomat MFT by Coviant Software is a secure, reliable managed file transfer solution designed to simplify and automate SFTP, FTPS, and HTTPS file transfers. Built for seamless integration, Diplomat MFT works across major cloud storage platforms, including AWS S3, Azure Blob, Google Cloud, Oracle Cloud, SharePoint, Dropbox, Box, and more.
-
5
Xtuner
A Next-Generation Training Engine Built for Ultra-Large MoE Models
...The framework focuses on enabling scalable training for extremely large models while maintaining efficiency across distributed computing environments. Unlike traditional 3D parallel training strategies, XTuner introduces optimized parallelism techniques that simplify scaling and reduce system complexity when training massive models. The engine supports training models with hundreds of billions of parameters and enables long-context training with sequence lengths reaching tens of thousands of tokens. Its architecture incorporates memory-efficient optimizations that allow researchers to train large models even when computational resources are limited. ... -
6
Sandstorm
One API call, pull Claude agent, completely sandboxed
...This approach lowers the friction of building autonomous agents by removing the need to provision servers, orchestrate distributed agents, or manage persistent tooling; agents can be spun up in parallel without manual setup and shut down when complete. The sandbox environment isolates agent execution for security and predictability, and project updates continue to harden observability, fault handling, and configuration validation. -
7
Ray
A unified framework for scalable computing
Modern workloads like deep learning and hyperparameter tuning are compute-intensive and require distributed or parallel execution. Ray makes it effortless to parallelize single machine code — go from a single CPU to multi-core, multi-GPU or multi-node with minimal code changes. Accelerate your PyTorch and Tensorflow workload with a more resource-efficient and flexible distributed execution framework powered by Ray. Accelerate your hyperparameter search workloads with Ray Tune. ... -
8
Stanza
Stanford NLP Python library for many human languages
...It contains tools, which can be used in a pipeline, to convert a string containing human language text into lists of sentences and words, to generate base forms of those words, their parts of speech and morphological features, to give a syntactic structure dependency parse, and to recognize named entities. The toolkit is designed to be parallel among more than 70 languages, using the Universal Dependencies formalism. Stanza is built with highly accurate neural network components that also enable efficient training and evaluation with your own annotated data. -
9
Nevergrad
A Python toolbox for performing gradient-free optimization
Nevergrad is a Python library for derivative-free optimization, offering robust implementations of many algorithms suited for black-box functions (i.e. functions where gradients are unavailable or unreliable). It targets hyperparameter search, architecture search, control problems, and experimental tuning—domains in which gradient-based methods may fail or be inapplicable. The library provides an easy interface to define an optimization problem (parameter space, loss function, budget) and... -
Download the most trusted enterprise browserChrome Enterprise is ideal for businesses of all sizes, IT professionals, and organizations looking for a secure, scalable, and easily managed browser solution that supports remote work, data protection, and streamlined enterprise operations.
-
10
Trafilatura
Python & command-line tool to gather text on the Web
Trafilatura is a Python package and command-line tool designed to gather text on the Web. It includes discovery, extraction and text-processing components. Its main applications are web crawling, downloads, scraping, and extraction of main texts, metadata and comments. It aims at staying handy and modular: no database is required, the output can be converted to various commonly used formats. Going from raw HTML to essential parts can alleviate many problems related to text quality, first by... -
11
BeeAI Framework
Build production-ready AI agents in both Python and Typescript
...BeeAI also provides orchestration tools for designing dynamic workflows, enabling multiple agents to coordinate tasks through structured execution flows, retries, and parallel processing. -
12
LangExtract
A Python library for extracting structured information
...LangExtract supports a wide range of models, including Google Gemini, OpenAI GPT, and local LLMs via Ollama, making it adaptable to different deployment environments and compliance needs. The system excels at handling long documents using optimized chunking, multi-pass extraction, and parallel processing to ensure both high recall and structured consistency. -
13
DeepEP
DeepEP: an efficient expert-parallel communication library
DeepEP is a communication library designed specifically to support Mixture-of-Experts (MoE) and expert parallelism (EP) deployments. Its core role is to implement high-throughput, low-latency all-to-all GPU communication kernels, which handle the dispatching of tokens to different experts (or shards) and then combining expert outputs back into the main data flow. Because MoE architectures require routing inputs to different experts, communication overhead can become a bottleneck — DeepEP... -
14
Colossal-AI
Making large AI models cheaper, faster and more accessible
...However, distributed training, especially model parallelism, often requires domain expertise in computer systems and architecture. It remains a challenge for AI researchers to implement complex distributed training solutions for their models. Colossal-AI provides a collection of parallel components for you. We aim to support you to write your distributed deep learning models just like how you write your model on your laptop. -
15
ERNIE
The official repository for ERNIE 4.5 and ERNIEKit
...It supports both full-parameter training and parameter-efficient approaches so teams can choose between maximum quality and lower-cost adaptation depending on their constraints. The project also emphasizes optimization techniques for large-scale training, including mixed-precision and hybrid-parallel strategies that are commonly needed for multi-node GPU clusters. In addition to training, it includes guidance and example materials intended to help developers adopt ERNIE models for real product scenarios rather than only research demonstrations. -
16
Meta-World
Collections of robotics environments
...The environments adhere to the Gymnasium API, which makes them easy to plug into existing RL pipelines, and they support both synchronous and asynchronous vectorized execution for running many environments in parallel. Installation is done via pip, with official support for Python versions 3.8 through 3.11 on Linux and macOS, and the project is licensed under MIT to encourage broad academic and industry use. -
17
TorchRec
Pytorch domain library for recommendation systems
...Parallelism primitives that enable easy authoring of large, performant multi-device/multi-node models using hybrid data-parallelism/model-parallelism. The TorchRec sharder can shard embedding tables with different sharding strategies including data-parallel, table-wise, row-wise, table-wise-row-wise, and column-wise sharding. The TorchRec planner can automatically generate optimized sharding plans for models. Pipelined training overlaps dataloading device transfer (copy to GPU), inter-device communications (input_dist), and computation (forward, backward) for increased performance. ... -
18
Transcrypt
Python 3.7 to JavaScript compiler
...As the project caught on and the number of people contributing issues, ideas and code grew, the repo was transferred to the QQuick organization, to be able to form a developer team on GitHub. There's a simple parallel between the Python and the JavaScript code. In combination with the use of source maps, this enables efficient debugging. Also, code can be tested from the command prompt using stubs. Lightning-fast JavaScript 6 code: call caching, for-loop optimization, in-line JavaScript etc. Integrated static typechecking and minification at the tip of a command-line switch. ... -
19
MiniMax-01
Large-language-model & vision-language-model based on Linear Attention
...MiniMax-Text-01 uses a hybrid attention architecture that blends Lightning Attention, standard softmax attention, and Mixture-of-Experts (MoE) routing to achieve both high throughput and long-context reasoning. It has 456 billion total parameters with 45.9 billion activated per token and is trained with advanced parallel strategies such as LASP+, varlen ring attention, and Expert Tensor Parallelism, enabling a training context of 1 million tokens and up to 4 million tokens at inference. MiniMax-VL-01 extends this core by adding a 303M-parameter Vision Transformer and a two-layer MLP projector in a ViT–MLP–LLM framework, allowing the model to process images at dynamic resolutions up to 2016×2016. -
20
higgsfield
Fault-tolerant, highly scalable GPU orchestration
Higgsfield is an open-source, fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters, such as Large Language Models (LLMs). -
21
Django friendly finite state machine
Django friendly finite state machine support
Django-fsm adds simple declarative state management for Django models. If you need parallel task execution, view, and background task code reuse over different flows - check my new project Django-view flow. Instead of adding a state field to a Django model and managing its values by hand, you use FSMField and mark model methods with the transition decorator. These methods could contain side effects of the state change. -
22
TextTest is an application-independent tool for text-based functional testing. This means running a batch-mode binary in lots of different ways, and using the text output produced as a means of controlling the behaviour of that application.
-
23
Parallel WaveGAN
Unofficial Parallel WaveGAN
Parallel WaveGAN is an unofficial PyTorch implementation of several state-of-the-art non-autoregressive neural vocoders, centered on Parallel WaveGAN but also including MelGAN, Multiband-MelGAN, HiFi-GAN, and StyleMelGAN. Its main goal is to provide a real-time neural vocoder that can turn mel spectrograms into high-quality speech audio efficiently. -
24
SCons
A software construction tool
SCons is a software construction tool that is a superior alternative to the classic "Make" build tool that we all know and love. SCons is implemented as a Python script and set of modules, and SCons "configuration files" are actually executed as Python scripts. This gives SCons many powerful capabilities not found in other software build tools. We make SCons available in three distinct packages, for different purposes. - The scons package is the basic package to install SCons. You... -
25
iX
Autonomous GPT-4 agent platform
...IX provides a flexible and scalable solution for delegating tasks to AI-powered agents. Agents created with the platform can automate a wide variety of tasks while running in parallel and communicating with each other.
