diff --git a/404.html b/404.html index 3046cf1..20aa49e 100644 --- a/404.html +++ b/404.html @@ -419,7 +419,7 @@

Content not found.

© 2024 Carsten Felix Draschner, PhD

- + diff --git a/about/index.html b/about/index.html index dc8a3ce..7c9b9a1 100644 --- a/about/index.html +++ b/about/index.html @@ -433,7 +433,7 @@

© 2024 Carsten Felix Draschner, PhD

- + diff --git a/blog/240202_llama2_13b_is_faster_than_llama2_7b/index.html b/blog/240202_llama2_13b_is_faster_than_llama2_7b/index.html new file mode 100644 index 0000000..de25bca --- /dev/null +++ b/blog/240202_llama2_13b_is_faster_than_llama2_7b/index.html @@ -0,0 +1,627 @@ + + + + + + LLAMA2 13B is faster than LLAMA2 7B, according to NVIDIA benchmark! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

LLAMA2 13B is faster than LLAMA2 7B, according to NVIDIA benchmark!

+ +
    +
    • +
  • blog
    • +
+ + +

Interesting findings on NVIDIA's LLAMA 2 benchmark results.

+

Image 1

+

TL;DR ⏱️

+
    +
  • NVIDIA LLAMA 2 Benchmark results
    • +
  • LLAMA 13B reported faster than LLAMA 7B
    • +
  • Questions about the accuracy of these findings
    • +
  • Seeking community insights
    • +
+

GenAI community/NVIDIA: I am confused! Can anyone help?

+

Interesting Findings 📈

+
    +
  • NVIDIA LLAMA 2 Benchmark (including sentence throughput)
    • +
  • Compares LLAMA-2 7B, 13B, and 70B
    • +
  • Weird finding: LLAMA 13B is reported to be faster than LLAMA 7B
    • +
  • Explicit Numbers: 7B Model has ~4 sentences/second throughput, 13B Model has ~7 sentences/second (LLAMA 70B ~1 sentence/second - this last one suits my expectation)
    • +
+

Questions 🤔

+
    +
  • NVIDIA NVIDIA AI, is there a mistake or can anyone else help me understand these numbers?
    • +
+

How we found out 📚

+
    +
  • Within our lovely GenAI team @Comma Soft AG, we are looking into tech details to implement the best solution
    • +
+ + +

Credit ❤️

+
    +
  • Nvidia thanks for providing benchmarks for LLAMA2
    • +
+

For further GenAI and ML tech discussions or such "weird" findings, reach out to me/follow me

+

#genai #machinelearning #llama

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240205_evil_llms_available_break_genai/index.html b/blog/240205_evil_llms_available_break_genai/index.html new file mode 100644 index 0000000..2de9a94 --- /dev/null +++ b/blog/240205_evil_llms_available_break_genai/index.html @@ -0,0 +1,645 @@ + + + + + + Evil LLMs available! Break GenAI Alignment through finetuning! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Evil LLMs available! Break GenAI Alignment through finetuning!

+ +
    +
    • +
  • blog
    • +
+ + +

Need for LLM Alignment transparency?

+

Image 1

+

TL;DR ⏱️

+
    +
  • Powerful LLMs are mostly aligned
    • +
  • Alignment can be broken through finetuning
    • +
  • Need for transparency in alignment processes
    • +
  • Questions about alignment in LLMs
    • +
+

For one of the most interesting open source LLMs, the Mixtral 8x7B, a finetuned LLM is available which has “broken” Alignment & answers to problematic prompts without prompt injections. Example in images (reference see below) shows “funny” but the astonishing LLM capabilities with broken Alignment.

+

Powerful LLMs are mostly aligned (Mixtral, LLAMA2, GPT4, …)

+
    +
  • They try to not give problematic responses
    • +
  • Some prompt-based attacks are already known to breach this behavior
    • +
  • But: model weights can be finetuned to break Alignment
    • +
  • Some use cases might need different Alignment than pre-implemented LLM Alignment or our standards are not reflected within LLM behavior.
    • +
  • Alignment process is majorly intransparent
    • +
+

LLM/GPT creation three-step approach ⚙️

+
    +
  1. Initial pretuning: Next token prediction
    2. +
  2. Chat/Instruction finetuning: training for conversational interaction & execution of tasks
    3. +
  3. Alignment: Adjust answers to not respond to critical questions like: creation of hate speech, critical advice in health issues, creation of spam or fraudulent content, and other
    4. +
+

Alignment Explanation 👩🏽‍🏫

+
    +
  • Done in a mixture of click workers (ethical aspects raised in linked article*) and AI as evaluator (RLHF/RLAIF). Rate which answers are better not to be given or should be given differently. Based on feedback model weights are adjusted.
    • +
  • Mostly intransparent process
    • +
  • Unknown what is truly covered (not) to be answered
    • +
+

My Questions 🤷🏼‍♂️

+
    +
  • Do you had ever issues with Alignment in LLM interaction?
    • +
  • Do you check Alignment when selecting an OS Model?
    • +
  • Have you ever adjusted Alignment on model weights basis?
    • +
  • Do you think it is valuable or too critical to release more or less aligned LLMs?
    • +
  • Do we need regulation for model alignment?
    • +
+

IMHO 🤗

+
    +
  • We need transparent statements on how models were aligned and how their behavior has changed, while covering ethical concerns when providing LLMs with reduced Alignment.
    • +
  • We need information on how easily well-adapted LLMs can be tripped with prompt engineering or finetuning.
    • +
  • We might need less aligned LLMs for research or in special use cases: +e.g. if in the healthcare sector a model should respond because an expert is using it as assistance, or for security reasons to create e.g. sample datasets for countermeasures against LLM-based phishing attacks (which are based on de-aligned) LLMs
    • +
  • Release models with awareness of possible dual use!
    • +
+

Within a great team @Comma Soft AG we are evaluating, selecting and finetuning open source LLMs for dedicated use cases.

+

Credit to: +Eric Hartford & Hugging Face & Mistral AI +https://lnkd.in/eyBSi4iu +AI Ethics - clickworkers: +https://lnkd.in/eKFfQZfF

+

#genai #artificialintelligence #aiethics #huggingface #llm #alignment

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240213_dalle_has_surprising_guardrails/index.html b/blog/240213_dalle_has_surprising_guardrails/index.html new file mode 100644 index 0000000..44ba1e2 --- /dev/null +++ b/blog/240213_dalle_has_surprising_guardrails/index.html @@ -0,0 +1,634 @@ + + + + + + DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes

+ +
    +
    • +
  • blog
    • +
+ + +

Interesting findings on DALLE's content filtering mechanisms.

+

Image 1

+

TL;DR ⏱️

+
    +
  • DALLE-3 filters your content AFTER image creation
    • +
  • With prompt “dead cookies” you can reproduce inconsistent filtering over OpenAI API
    • +
  • 40% of cases with same “dead cookies” prompt stop through content filter and 60% reach us over API
    • +
+

What is DALLE-3 🖼️

+
    +
  • DALLE 3 is a generative text-to-image model by OpenAI also available as API
    • +
  • You pay per image
    • +
  • Images are created based on your prompt like “dead cookies”.
    • +
  • You can also add details like: “Dead Cookies in cute Pixar style” or “Dead cookies with dramatic situation in cute Pixar style”
    • +
  • Open-Source image GenAI models alternatives are available e.g., Stable Diffusion
    • +
  • Image GenAI are under discussion because of misuse like deepfakes or reproducing intellectual property.
    • +
+

Finding/Observation: 👩🏽‍🔬

+
    +
  • DALLE-3 has a content filter to reduce misuse
    • +
  • If you hit the content filter you do not get a resulting image for your prompt.
    • +
  • The content filter is not applied based on the prompt, it is applied AFTER DALLE-3 generated the image, and the API decides in an extra step if the image should be sent to you. Likely some Image classifier.
    • +
  • Same prompt sometimes results in an image and sometimes in a content-filter response. For the prompt “dead cookies” you get in 60% of requests an image and in 40% a content filter issue
    • +
+

How we found out 🍪

+
    +
  • We @Comma Soft AG develop tools and pipelines with OS GenAI but also with API requests.
    • +
  • For good API-response handling, we also had to consider content filter scenarios. So we combined trigger words like "dead" with something like "cookies"
    • +
  • We had inconsistent content filter and still the finding that in case of content filter the response time was roughly as long as in the case of created image.
    • +
+

My Questions to you 🤷🏼‍♂️

+
    +
  • Who should pay for “dead cookies” if the resulting image was created but not sent due to content filter?
    • +
  • Have you known that the content filter for DALLE-3 is applied after image generation?
    • +
  • Do you also encounter content filter although your prompts were in principle ok?
    • +
  • Do you think content filters are a reasonable image GenAI misuse countermeasure?
    • +
  • How would you reduce Image GenAI misuse?
    • +
  • And most interesting (and we might never know OpenAI/DALL-E Open Ai), how do “Dead Cookies” images look like which are filtered out? 😅
    • +
+

The image was created by the prompt "Dead cookies in cute pixar style" +If you like more of such content, reach out to me 😊

+

#artificalintelligence #genai #aiethics #dalle #openai #texttoimage #deepfakes

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240227_ever_wondered_about_open_source_llm_sizes/index.html b/blog/240227_ever_wondered_about_open_source_llm_sizes/index.html new file mode 100644 index 0000000..91d81c2 --- /dev/null +++ b/blog/240227_ever_wondered_about_open_source_llm_sizes/index.html @@ -0,0 +1,638 @@ + + + + + + Ever wondered about open source LLLM sizes - 7B, 13B, 70B? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Ever wondered about open source LLLM sizes - 7B, 13B, 70B?

+ +
    +
    • +
  • blog
    • +
+ + +

Where do those model sizes come from?... My findings!

+

Image 1

+

TL;DR ⏱️

+
    +
  • Open-source LLM alternatives to AIaaS
    • +
  • Hugging Face as a source for open-source models
    • +
  • Many models are finetuned variants
    • +
  • Bigger models imply slower inference & higher costs
    • +
  • Different use cases require different model capabilities
    • +
  • Questioning the parameter step sizes of models
    • +
+

Background 📝

+
    +
  • As an alternative to AIaaS like ChatGPT, you can interact with LLMs based on open-source models.
    • +
  • A good source for open-source models is the Hugging Face model library.
    • +
  • Many models are finetuned variants of existing models like the Meta LLAMA-2 is available as 7B, 13B, 70B.
    • +
  • Inference runs faster on GPUs like NVIDIA V/A/H100.
    • +
  • Bigger models: slower inference & have higher (environmental) costs while bigger LLMs mostly outperform smaller models in benchmark-tasks.
    • +
  • Within multiple use cases, I select the best fitting OS model at @Comma Soft AG use cases.
    • +
  • Different use cases request different model capabilities including model “knowledge” or inference speed.
    • +
  • I was wondering why many models follow the parameter “step sizes” 7B, 13B, and 70B.
    • +
+

My Findings 🔍

+
    +
  • Many models are finetuned versions of LLAMA-2 as this was a high-performing open-source LLM available within a “mostly” attractive OS license.
    • +
  • In most cases, model finetuning does not change the number of parameters.
    • +
  • LLAMA paper states it provides LLMs: [...] "that achieve the best possible performance at various inference budgets" [...]
    • +
  • Common hardware is 16GB or 80GB of VRAM. Usually, you have one or two of those GPUs within a system.
    • +
  • Models are by default available as 16bit representation which leads to 2byte per parameter.
    • +
  • To run a model, you need space for parameters and a bit remaining for your batch. So 7B fits on 1x 16GB GPU, 13B fits on 2x 16GB GPUs, (the Lab-leaked LLAMA-1 with 35B fits on 1x 80GB GPU) and the 70B LLAMA-2 model runs on 2x 80GB GPUs.
    • +
+

IMHO 🤗

+
    +
  • I am looking forward to how 4bit quantization like GPTQ or AWQ changes the model sizes as you might also fit a roughly 145B quantized model on a single A100 with 80GB.
    • +
  • Some use cases might need fewer model parameters but bigger batches, longer max context length, or faster inference which means fewer parameters or fewer deep networks.
    • +
  • Consider smaller models especially because of the environmental costs if performance is sufficient.
    • +
  • I am wondering if there is a true reason for how the parameters are combined within the architecture, the numbers partially feel randomly picked like 80 transformer layers for LLAMA-2 70B vs 40 of 13B version.
    • +
+

Questions 🤔

+
    +
  • What is your preferred model(-family)?
    • +
  • Do you use your models as plain or quantized versions?
    • +
  • Do you think the model architectures of finetuned context window, hidden size, intermediate size, number heads, and transformer layers are well chosen that build the total needed VRAM volume?
    • +
+

Follow me for more content ❤️

+

#artificialintelligence #genai #machinelearning #llm

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240307_not_prompting_in_english/index.html b/blog/240307_not_prompting_in_english/index.html new file mode 100644 index 0000000..d27512b --- /dev/null +++ b/blog/240307_not_prompting_in_english/index.html @@ -0,0 +1,634 @@ + + + + + + Not prompting in English?... You have Problems!! LLM Language Barriers • Democratizing GenAI and fair pricing + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Not prompting in English?... You have Problems!! LLM Language Barriers • Democratizing GenAI and fair pricing

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the challenges of using Generative AI with languages other than English and the implications for cost and performance.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Tokenizers and their role in LLMs
    • +
  • Challenges of non-English prompts
    • +
  • Efficiency and fairness in GenAI
    • +
  • Recommendations for LLM pipelines
    • +
+

Tokenizer ✂️

+
    +
  • Generative AI Large Language Models process tokens
    • +
  • Tokenizers are a preprocessing step of LLMs and cut prompts into subwords called tokens
    • +
  • Tokenizers are optimized to efficiently compress text into a limited number of tokens
    • +
  • Tokenizers are optimized based on large text corpora
    • +
  • LLM pipeline hardware load grows based on number of tokens
    • +
  • OpenAI GPT API charges per token
    • +
+

My findings & my takes 😰

+
    +
  • You do not prompt in English... You have Problems!
    • +
  • You have to pay more if you are communicating in a language other than English as the tokenizer creates more tokens (see image)
    • +
  • You'll likely get worse results from LLMs since they haven't seen as much text in your target language during pretraining
    • +
  • You can fit less non-English text into an LLM, which has limited context length, meaning you can't solve similarly complex tasks with same LLM
    • +
  • Using LLMs in German I am still quite lucky since German is reasonably well represented in most training data corpora and due to the relative similarity between German and English the difference in token decompositions is less pronounced. However, there are many languages (in particular those not using the Latin alphabet) for which the effects are much larger.
    • +
+

IMHO 🤗

+
    +
  • We will gather extraordinary efficiency gains through generative AI but we need to be careful that these gains are available to truly all people including those not using GenAI in English
    • +
  • If you are building your own LLM pipeline as we do at Comma Soft AG, check out how the tokenizer and language model interact with your language. This might affect the costs or max context length of your text
    • +
  • As the pricing per token is based on the technical costs to process a certain text, I can understand it from that perspective… But it is unpleasant as you pay more for the same prompt while likely getting worse results and having more limitations in prompt complexity e.g. when using RAG insertions
    • +
  • Possibly LLMaaS could provide differently trained LLMs focused on regions and languages yielding better tokenizers and performance for lower resource languages
    • +
+ +
    +
  1. Who should decide about alignment? especially when we speak about regional LLMs: https://shorturl.at/dwGIQ
    2. +
  2. What is fair pricing for content-filtered GenAI results: https://shorturl.at/aCF16
    3. +
  3. What is the best LLM for you: https://shorturl.at/bmuP8
    4. +
  4. Try out Open AI tokenizer: https://lnkd.in/eKhrP7qi
    5. +
+

Hashtag#generativeai Hashtag#artificialintelligence Hashtag#machinelearning Hashtag#aiethics

+

For more content, brainstorming, and discussions, follow me or reach out to me ❤️

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240313_the_major_players_in_genai_are_facing_challenges/index.html b/blog/240313_the_major_players_in_genai_are_facing_challenges/index.html new file mode 100644 index 0000000..7816391 --- /dev/null +++ b/blog/240313_the_major_players_in_genai_are_facing_challenges/index.html @@ -0,0 +1,623 @@ + + + + + + The major players in GenAI are facing challenges with their Generative AIs. GenAI capabilities and security issues related to LLMs Tools • 37C3 Presentation + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

The major players in GenAI are facing challenges with their Generative AIs. GenAI capabilities and security issues related to LLMs Tools • 37C3 Presentation

+ +
    +
    • +
  • blog
    • +
+ + +

Challenges and security issues in GenAI and LLMs, highlighted at 37C3.

+

Image 1

+

TL;DR ⏱️

+
    +
  • GenAI has immense capabilities
    • +
  • Ethical and secure GenAI pipelines are crucial
    • +
  • 37C3 presentation on security issues and exploitations
    • +
  • Categories of threats and challenges in GenAI
    • +
+

Introduction 📖

+

GenAI has immense capabilities and can support a variety of processes. As developers, we have the responsibility to build ethical and secure GenAI pipelines. A noteworthy but easily overlooked presentation at the 37C3 event highlighted how tools from leading companies like OpenAI, Microsoft, and Google have been exploited through "Indirect Prompt Injections".

+

Threat Categories ⚠️

+
    +
  • Model Issues: Bias, offensive/dangerous responses, hallucinations, backdoored models.
    • +
  • User as the attacker: Direct Prompt Injection, Print/Overwrite System Instructions Do-Anything-Now, Denial of Service.
    • +
  • Indirect Prompt Injections: AI Injection, Scams, Data Exfiltration, Plugin Request Forgery.
    • +
+

Challenges 💪🏼

+

The development of security and safeguards for GenAI is still a "work in progress" and consists of various components 👮🏼

+
    +
  • Alignment is a fine-tuning approach aimed at training model behaviors to respond securely to critical prompts. Learn more here: https://lnkd.in/eWS-VZCD
    • +
  • In Image GenAI, for instance, you can use a post-processing content filter. Check out this peculiar content filter behavior here: https://lnkd.in/eRaX9JuT
    • +
  • An interesting set of additional threats and potential countermeasures were recently presented at the 37C3 event, focusing on Indirect Prompt Injections.
    • +
+ +
    +
  • If you're interested in this topic, especially Indirect Prompt Injections, I highly recommend the 37C3 presentation. Watch the video on YouTube: https://lnkd.in/enjMee6S
    • +
  • A huge thanks to everyone supporting the security of these GenAI tools and for their tireless efforts in educating and presenting their findings for free. Special thanks to Johann Rehberger
    • +
  • As part of an exceptional team at @Comma Soft AG, we're developing pipelines with a holistic perspective on GenAI.
    • +
+

#generativeai #llm #aisecurity #aiethics #37c3

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240327_be_careful_when_you_are_using_llama/index.html b/blog/240327_be_careful_when_you_are_using_llama/index.html new file mode 100644 index 0000000..9599063 --- /dev/null +++ b/blog/240327_be_careful_when_you_are_using_llama/index.html @@ -0,0 +1,638 @@ + + + + + + Be careful when you are using LLAMA-2! Legal risks & Sustainability Implications due to LLAMA-2 is (NOT) Open Source. + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Be careful when you are using LLAMA-2! Legal risks & Sustainability Implications due to LLAMA-2 is (NOT) Open Source.

+ +
    +
    • +
  • blog
    • +
+ + +

Important considerations regarding LLAMA-2's legal and sustainability implications.

+

Image 1

+

TL;DR ⏱️

+
    +
  • LLAMA-2's legal and sustainability challenges
    • +
  • Not truly open-source according to OSD
    • +
  • Technical implications of its license
    • +
  • Meta's restrictions and their broader impact
    • +
+

What is LLAMA-2? 🦙

+
    +
  • LLAMA 2 is a Large Language Model by Meta
    • +
  • Thanks to Meta its weights are openly available over e.g. Hugging Face
    • +
  • Meta claims to provide LLAMAs as open-source Models
    • +
  • LLAMA-2 is under the LLAMA license which has some restrictions [1]
    • +
+

Why is LLAMA NOT Open Source? [2] ❌

+
    +
  • Open Source means software under a license aligned with Open Source Definition (OSD)
    • +
  • This includes no discrimination against persons or groups or fields of endeavor (OSD points 5 and 6)
    • +
  • Meta’s license puts restrictions for commercial use (paragraph 2)
    • +
  • Meta also restricts the use of the model and software for certain purposes (the Acceptable Use Policy)
    • +
+

Technical Implications! 👨🏼‍💻

+

IMHO, From a technical perspective the license statement $1V is another major challenge: “You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof)”. I interpret this as a barrier if you use this model in combination with other LLMs in workflows like:

+
    +
  • AI as a judge in benchmark or evaluation pipelines
    • +
  • Reinforcement Learning with AI Feedback - RLAIF
    • +
  • Data Synthesis pipelines
    • +
  • Model Distillation
    • +
+

Meta creates issues with the “open source” LLAMA-2 😢

+
    +
  • We will not reach true democratization of LLM application when usage is limited like this
    • +
  • Those licenses can create confusion about what is open source in general, what is truly allowed, and what is not with those LLMs
    • +
  • If we can barely reuse those pre-tuned LLMs we waste a lot of energy/resources spent on pretraining on more or less the same internet text corpus. 🌱
    • +
  • And we still do not know how it is truly pre-trained, finetuned, and aligned. I’d appreciate more transparency here!
    • +
+

All of those statements are a part of our R&D journey @Comma Soft AG [3,4] developing LLM tools aligned with such challenging special licenses. I want to bring up the discussion about LLM usage and the meaning of open source and I am NOT giving legal advice.

+

What do you think about Meta's LLAMA-2 license? Will they change it in LLAMA-3? And what implications do you derive from this license?

+

Reading list 📖

+

[1] LLAMA-2 License
+[2] Meta’s LLaMa 2 license is not Open Source
+[3] Weird LLAMA benchmark
+[4] Reasons for LLAMA sizes

+

#artificialintelligence #machinelearning #llm #aiethics

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240408_too_many_llms_how_to_keep_track_with_all/index.html b/blog/240408_too_many_llms_how_to_keep_track_with_all/index.html new file mode 100644 index 0000000..ed6aa02 --- /dev/null +++ b/blog/240408_too_many_llms_how_to_keep_track_with_all/index.html @@ -0,0 +1,645 @@ + + + + + + Too many LLMs?! How to keep track with all the Open Source Models? Identify the finetuned-masked LLMs and its position within the GenAI landscape! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Too many LLMs?! How to keep track with all the Open Source Models? Identify the finetuned-masked LLMs and its position within the GenAI landscape!

+ +
    +
    • +
  • blog
    • +
+ + +

Navigating the complex landscape of GenAI models can be challenging, but it's crucial to understand the foundational and finetuned models to make informed decisions.

+

Image 1

+

TL;DR ⏱️

+
    +
  • The GenAI landscape is crowded with many models
    • +
  • Keeping track of innovations and true effects is hard
    • +
  • Transparency issues with many so-called "open-source" models
    • +
  • Recommendations for navigating this landscape
    • +
+

The current LLM landscape 🤖

+
    +
  • Big GenAI LLM competition
    • +
  • Lots of proprietary and open-source models
    • +
  • Many Competitors: Big players like Google, Meta, and Microsoft; rising companies like Mistral and OpenAI, but also smaller institutions with foundation models or finetuned models
    • +
  • Several release their model parameters openly on platforms like Huggingface
    • +
  • Number of GenAI models are 500k+ (HF total), 60k+ (HF text generation)
    • +
+

Challenges with number of LLMs 🤯

+
    +
  • Difficult to keep track with so many models
    • +
  • Most innovations are only documented with blog posts or arxiv papers
    • +
  • True innovations and effects are barely reported
    • +
  • Model leaderboards are flooded with finetuned models
    • +
  • Not always clear which and why model architecture and training data are combined
    • +
+

My problem with current landscape 🤦🏼‍♂️

+
    +
  • Many claim to provide open-source models but do not disclose how those are constructed. What are the true training data and initial training datasets for pretuning, instruction tuning, and alignment?
    • +
  • Which training hyperparameter setup was used and why
    • +
  • Why does a specific model have exactly this combination of parameters like: number transformer layers, number heads, hidden-size, intermediate size
    • +
+

My recommendation 🤗

+
    +
  • Keep calm and check what is the true source foundation model
    • +
  • Understand the architecture: e.g. more or less all of those are autoregressive Decoder-Only multilayer transformer networks
    • +
  • Some introduce a novel architecture ideas like MoE (Mixture of Experts)
    • +
  • Do your research, demystify LLM-landscape, and implement benchmarks you trust as we do @Comma Soft AG!
    • +
+

Further reading 📚

+ +

Credit ❤️

+
    +
  • R&D teams providing truly open-source models
    • +
  • Accessible research contributions explaining effects of LLM optimization techniques
    • +
  • LMSYS Chatbot Arena as an alternative to Open LLM leaderboard with focus on the original foundation models
    • +
+

How do you find the most promising OS foundation model for your UC?

+

For more content, brainstorming and discussions, follow me or reach out to me 🥰

+ + + + +
+ + + + + + + diff --git a/blog/240415_nvidia_benchmark_might_be_wrong/index.html b/blog/240415_nvidia_benchmark_might_be_wrong/index.html new file mode 100644 index 0000000..2fb3782 --- /dev/null +++ b/blog/240415_nvidia_benchmark_might_be_wrong/index.html @@ -0,0 +1,649 @@ + + + + + + NVIDIA Benchmark might be WRONG cause it states - You lose money AND LLM inference speed if you add more NVIDIA A100. This NVIDIA Benchmark is NOT reliable. + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

NVIDIA Benchmark might be WRONG cause it states - You lose money AND LLM inference speed if you add more NVIDIA A100. This NVIDIA Benchmark is NOT reliable.

+ +
    +
    • +
  • blog
    • +
+ + +

Analyzing the reliability of NVIDIA's benchmark results and the implications for LLM inference speed and hardware investment.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Terms and background on LLMs and inference
    • +
  • Strange findings in NVIDIA's benchmark results
    • +
  • Concerns about the reliability of these benchmarks
    • +
  • Questions and further reading on the topic
    • +
+

Terms 🏫

+
    +
  • LLMs are Large Language Models
    • +
  • LLMs are a branch of Generative AI
    • +
  • Those LLMs can be used to generate texts
    • +
  • Text generation by LLMs is called Inference
    • +
  • LLM Inference is faster on GPUs compared to CPUs
    • +
  • Pretty common “LLM-GPU” is the NVIDIA A100
    • +
+

Background Story ⚙️

+
    +
  • We @Comma Soft AG are developing LLM pipelines
    • +
  • Each use case has different requirements
    • +
  • Sometimes Inference Speed is more important
    • +
  • More Hardware performance can/should improve Inference speed
    • +
  • To check out how much you can improve with more hardware, you can look into the scaling effect to see the trade-off between Inference speed and hardware costs
    • +
+

Weird Finding 🤔

+
    +
  • NVIDIA released a benchmark (link see below)
    • +
  • It compares different GPU setups: 1, 2, 4, 8 GPUs for a common open-source model inference
    • +
  • It states that when you increase from 2 GPUs to 4 GPUs you get half the throughput; from 10 sentences/sec to 4.8 sentences/sec for LLAMA-2 13B
    • +
+

My Take 🤗

+
    +
  • The NVIDIA Benchmark is broken or some hiccup with copy-paste of results
    • +
  • Sentences/sec is a strange measure. Why not tokens per second which is more stable
    • +
  • I found another strange issue with model sizes and performance on NVIDIA GPUs in this benchmark. see this link: https://rb.gy/5l8qqp
    • +
  • It is a problem when you cannot trust benchmarks as this leads to reimplementing benchmarks or running them again which is a waste of resources and barely sustainable
    • +
  • Benchmarks should be available open source to understand the measures and issues
    • +
+

Questions 🔠

+
    +
  • What do you think is the reason for this weird benchmark result?
    • +
  • Do you have an idea why they measure in sentences per second and not in tokens per second?
    • +
  • What are your preferred sources for benchmarks when it comes to Inference performance?
    • +
  • What do you do to improve inference speed?
    • +
+ + +

NVIDIA could you please fix it or comment on what was the issue/reason

+

For more content, brainstorming, and discussions, follow me or reach out to me 🥰

+

#LostInGenai #artificialintelligence #selectllm

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240422_what_is_your_preferred_llm_family/index.html b/blog/240422_what_is_your_preferred_llm_family/index.html new file mode 100644 index 0000000..89df1f5 --- /dev/null +++ b/blog/240422_what_is_your_preferred_llm_family/index.html @@ -0,0 +1,621 @@ + + + + + + What is your preferred LLM family? And do you start with an already finetuned LLM? Why you have chosen this LLM? I love to hear your perspective! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

What is your preferred LLM family? And do you start with an already finetuned LLM? Why you have chosen this LLM? I love to hear your perspective!

+ +
    +
    • +
  • blog
    • +
+ + +

Understanding the preferences and choices behind selecting specific LLM families and their finetuned variants.

+

Which Model

+

TL;DR ⏱️

+
    +
  • GenAI for text implemented by LLMs
    • +
  • Many open-source models available
    • +
  • Continuous influx of new models
    • +
  • Key foundation model families
    • +
  • LLM-based GenAI pipelines at Comma Soft AG
    • +
+

Background Information ⚙️

+
    +
  • GenAI for text can be implemented by LLMs
    • +
  • LLMs are partially open-source available
    • +
  • Day by day new models come to market or are published on platforms like Huggingface
    • +
  • Many of them rely on certain Foundation model families
    • +
  • Some of them exist in different sizes
    • +
  • We @Comma Soft AG develop LLM based GenAI pipelines (sometimes using OS models like the ones in the survey)
    • +
+

Further Reading 📚

+ +

If you need support with LLM R&D or simply want to chat, reach out to me and follow me for more content ❤️

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240422_who_will_take_care_of_truly_low_resource_languages/index.html b/blog/240422_who_will_take_care_of_truly_low_resource_languages/index.html new file mode 100644 index 0000000..4289dce --- /dev/null +++ b/blog/240422_who_will_take_care_of_truly_low_resource_languages/index.html @@ -0,0 +1,645 @@ + + + + + + Who will take care of truly low-resource languages? A good step towards more fair GenAI LLM pricing at OpenAI for Japanese-using people! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Who will take care of truly low-resource languages? A good step towards more fair GenAI LLM pricing at OpenAI for Japanese-using people!

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the challenges and recent developments in addressing low-resource languages within the GenAI landscape, with a focus on OpenAI's efforts for the Japanese language.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Issues with LLMs for low-resource languages
    • +
  • Major challenges with different character languages
    • +
  • OpenAI's new dedicated model for Japanese
    • +
  • Concerns about AI ethics and inequality
    • +
+

Recap 🔁

+
    +
  • In a recent post, I described the issues of LLM solutions with "low resource" languages (see link below)
    • +
  • Those issues are even more complex with languages using entirely different characters than the majorly used language English
    • +
+

The major issues ⚙️

+
    +
  • The same text needs more tokens to be represented
    • +
  • More tokens need more time to be generated and are more costly e.g. in the OpenAI API pricing
    • +
  • The quality of results is less good if a dedicated language is not in focus of model training.
    • +
  • RAG use cases and also bigger task descriptions are more limited as fixed context size might limit the expression of details and context
    • +
+

News by OpenAI 📰

+
    +
  • Dedicated Model for Japan
    • +
  • Focus on their tokens leading to improvements in quality and lowering effective costs (see link)
    • +
+

My Take 🤗

+
    +
  • I appreciate these efforts by Open AI
    • +
  • We might need more such efforts towards further democratizing LLMs to different language heritages.
    • +
+

My Major worry (AI Ethics) 😓

+
    +
  • GenAI will provide major gains in economic efficiency and productivity
    • +
  • The development of competitive models needs huge financial investments.
    • +
  • This leads to the fact that already rich countries with their languages will have access to those technologies and will over proportionally make use of those efficiency gains.
    • +
  • It is unclear to what extent low-resource languages with different characters are part of LLM training.
    • +
  • I fear that this can further improve inequality as the invested resources (electric energy, hardware production...) and its footprint will influence our whole planet but its gains once again only "our" privileged lives.
    • +
+

Credit 😍

+
    +
  • To OpenAI for tackling this "known" issue of unfair and inefficient handling Japanese Language
    • +
  • To all R&D Teams driving efforts towards low resource language GenAI
    • +
+

We @Comma Soft AG provide a B2B LLM as a Service with a focus on German and English Language and even though German is less complicated than other truly low-resource languages, it was already quite a challenge to reach high-quality results.

+ + +

Do you develop LLMs with a focus outside of the English Language?

+

For more content, follow me ❤️

+

#artificialintelligence #genai #llm #aiethics #japan #openai

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240506_thanks_to_the_open_source_community/index.html b/blog/240506_thanks_to_the_open_source_community/index.html new file mode 100644 index 0000000..b26bc80 --- /dev/null +++ b/blog/240506_thanks_to_the_open_source_community/index.html @@ -0,0 +1,621 @@ + + + + + + Thanks to the Open Source Community for all their efforts! Greetings from PyCon 2024 + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Thanks to the Open Source Community for all their efforts! Greetings from PyCon 2024

+ +
    +
    • +
  • blog
    • +
+ + +

Expressing gratitude to the open-source community and sharing experiences from PyConDE & PyData Berlin 2024.

+

PyCon 2024

+

TL;DR ⏱️

+
    +
  • Trip to PyCon with colleagues
    • +
  • Attended insightful talks in various AI fields
    • +
  • Appreciation for open-source community
    • +
  • Gratitude to all contributors and supporters
    • +
+

Trip to PyCon 🚅

+

With my great colleagues, Dr. Laura Maaßen and Sebastian Schönnenbeck, from our Data Science and GenAI R&D teams, we are excited to attend PyConDE & PyData Berlin.

+

Great Talks 🎤

+

We attended so many inspiring and insightful talks in the fields of Machine Learning, MLOps, GenAI, LLM, AI Ethics, and open-source software development.

+

My Take 🤗

+
    +
  • I greatly appreciate the efforts of the open-source development community!
    • +
  • Having all these open-source tools and freely available tech education is such a privilege!
    • +
  • I am so glad to be in a field where so much great material is accessible online for free.
    • +
  • At conferences like PyCon, you meet the people behind the projects, understand their challenges, and get inspired by their motivation that drives their efforts.
    • +
+

Credit ❤️

+
    +
  • To all the Open Source Developers for their efforts in creating so many great and helpful tools.
    • +
  • To all the open education supporters who create great material accessible to those eager to learn.
    • +
  • To the PyCon Organising Team and all the Volunteers making this event possible.
    • +
  • Thanks to Comma Soft AG for investing in R&D and funding us this trip with the opportunity to learn, discuss, and get inspired.
    • +
  • And finally, thanks to Laura and Sebastian who supported me during the conference ❤️‍🩹. I needed help after knee surgery being at PyCon on crutches and equipped with a knee brace (sports can be dangerous 😉).
    • +
+

#PyCon #opensource #artificialintelligence #freeeducation

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240513_be_careful_when_you_speak_of_open_source_genai/index.html b/blog/240513_be_careful_when_you_speak_of_open_source_genai/index.html new file mode 100644 index 0000000..0628378 --- /dev/null +++ b/blog/240513_be_careful_when_you_speak_of_open_source_genai/index.html @@ -0,0 +1,632 @@ + + + + + + Be careful when you speak of Open (Source) GenAI. Why OpenAI and Meta (shouldn't) use the word Open within their GenAI efforts? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Be careful when you speak of Open (Source) GenAI. Why OpenAI and Meta (shouldn't) use the word Open within their GenAI efforts?

+ +
    +
    • +
  • blog
    • +
+ + +

Examining the implications of using the term "Open" in the context of GenAI by organizations like OpenAI and Meta.

+

Image 1

+

TL;DR 🚅

+
    +
  • Open Source is a huge and important field in computer science and AI
    • +
  • The word "Open" is used widely within the GenAI field: OpenAI, Open Source LLMs
    • +
+

Background of term open in OpenAI and Open (Source) LLMs 🧑🏽‍🏫

+
    +
  • Open source libraries accelerate and improve the IT field including GenAI
    • +
  • Many GenAI models are available on model hubs like Hugging Face
    • +
  • Often those models like Meta LLAMA-3 are called open-source LLMs
    • +
  • OpenAI provides the most successful LLM GPT4
    • +
  • OpenAI does not release their model weights or creation details openly
    • +
  • Elon Musk being a former founder of OpenAI sued them (maybe also cause of other reasons) for being less open than initially founded for
    • +
+

Why OpenAI and Meta think it is okay to use the term "Open" 🤓

+
    +
  • Sam Altman CEO of OpenAI claims in an interview that "Open" is valid as ChatGPT made GenAI/LLMs accessible to the world openly
    • +
  • Meta claims open source as their LLM LLAMA-3 weights are freely downloadable
    • +
+

Issues 😓

+
    +
  • No one could reproduce OpenAI or Meta's Models, so they are not open-source
    • +
  • This missing open source and open research is an issue for reproducibility, fairness, security, democratization of tech, and environmental impact
    • +
+

My Take (IMHO) 🤗

+
    +
  • OpenAI can take credit for the hype around LLM-based GenAI as they truly opened access to this tool through a (partial) free open chat interface.
    • +
  • Most impactful models on HuggingFace like Meta's LLAMA-3 should be considered being OPEN WEIGHT and not open source. Also, there are sometimes issues with its licenses: https://lnkd.in/dtStQSdn
    • +
  • Truly open source would mean code can entirely reproduce the models
    • +
  • Still, these open-weight models accelerate GenAI/LLM developments
    • +
+

Questions? 🤔

+
    +
  • How do you see the possible confusion of the word "open" initially coming from open-source in GenAI efforts of OpenAI and Meta LLAMA-3 open-weight model?
    • +
  • Do you use any truly open-source LLM?
    • +
+

For more content, follow me or our team at Comma Soft AG on LinkedIn ❤️

+

#artificialintelligence #llm #opensource #llama #genai

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240520_what_expectations_do_you_have_regarding_the_values_and_norms/index.html b/blog/240520_what_expectations_do_you_have_regarding_the_values_and_norms/index.html new file mode 100644 index 0000000..73b6a62 --- /dev/null +++ b/blog/240520_what_expectations_do_you_have_regarding_the_values_and_norms/index.html @@ -0,0 +1,633 @@ + + + + + + What expectations do you have regarding the values and norms of your GenAI chat assistants? Highly sensitive topic in the LLM space! My take... + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

What expectations do you have regarding the values and norms of your GenAI chat assistants? Highly sensitive topic in the LLM space! My take...

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the ethical considerations and expectations surrounding the values and norms embedded in GenAI chat assistants.

+

Image 1

+

TL;DR ⏱️

+
    +
  • LLMs generate text based on training
    • +
  • Alignment and finetuning influence behavior
    • +
  • Ethical considerations in different languages
    • +
  • Need for a holistic view on model behavior
    • +
+

Background Information 🤓

+
    +
  • LLMs are GenAI models that generate texts
    • +
  • The text-generating behavior is based on the LLM's training
    • +
  • Chat LLMs are trained in three phases
    • +
  • Pretuning on large (filtered) text corpora
    • +
  • Finetuning on, for example, instruction tuning data
    • +
  • Final alignment to encourage preferred responses
    • +
+

Some classic methods for adapting LLM behavior 👮🏼

+
    +
  • Filter certain content, for example, NSFW, from the pretuning data
    • +
  • Try to induce certain behaviors that we consider appropriate through alignment
    • +
  • Craft system prompts in which we try to give the Chat Agent specific character traits
    • +
+

Possible Issues 🫣

+
    +
  • Would you expect different values and norms depending on the language in which you are prompting? Apart from alignment, culture and language may correlate with each other in the respective token spaces even in pretraining.
    • +
  • Do LLMs map all languages and the values they contain into different or the same space?
    • +
  • When using LLMs, we should not only pay attention to the performance of the models but also to the behavior (in other languages and token spaces)
    • +
  • Does alignment-finetuning in one language resolve subjectively problematic behavior in other prompt languages?
    • +
  • Who should decide about norms and values embedded within GenAI approaches? And do you check those details when selecting a pretuned model?
    • +
+

Further Reading 📖

+ +

We at Comma Soft AG develop Large Language Models and GenAI pipelines with a holistic perspective on model behavior and performance. Please share your favorite papers or simply your experiences with this topic! For more AI topics outside the daily random model updates, please get in contact with me or follow me here on LinkedIn. ❤️

+

#LostInGenai #artificialintelligence #selectllm

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240603_do_you_differentiate_ai_ethics/index.html b/blog/240603_do_you_differentiate_ai_ethics/index.html new file mode 100644 index 0000000..4b731e7 --- /dev/null +++ b/blog/240603_do_you_differentiate_ai_ethics/index.html @@ -0,0 +1,629 @@ + + + + + + Do you differentiate AI Ethics principles between AI/ML fields like GenAI/LLMs or Knowledge Graph-based ML? How do we deal with so-called facts on the internet as training data? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Do you differentiate AI Ethics principles between AI/ML fields like GenAI/LLMs or Knowledge Graph-based ML? How do we deal with so-called facts on the internet as training data?

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the nuances of AI ethics across different AI/ML fields and handling internet-based training data responsibly.

+

Ethics in AI

+

TL;DR ⏱️

+
    +
  • AI ethics principles across different AI/ML fields
    • +
  • Personal background and perspective on AI ethics
    • +
  • Recommendations and further reading on AI ethics
    • +
  • Questions to ponder on AI ethics practices
    • +
+

My Background and Perspective 👶

+
    +
  • AI ethics is always the same. No matter if GenAI, Computer Vision, or Knowledge Graph-based ML!
    • +
  • In recent years, I had three main focus areas in the field of AI: My Ph.D. passion in Knowledge Graph-based Machine Learning, before a focus on computer vision, and most recently GenAI with a focus on text
    • +
  • It was always important for me to take a holistic approach. This includes ethical and sustainability aspects in particular.
    • +
  • In our paper on Ethical and Sustainability Implications for Knowledge Graph-based Machine Learning, we have elaborated on topics I also recently found in the literature on LLM AI Ethics [1,2].
    • +
  • I see a chance that AI areas majorly align in the AI Ethics dimensions, so we can easily learn from each other and share best practices
    • +
+

Recommendation 👍🏽

+
    +
  • At this point, I recommend a well-known but still very valid paper as an introduction to LLM ethics for those who may not yet know it or are new to the field: On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell [1]
    • +
+

Further Reading 📖

+ +

I hope that all those who develop such powerful tools are aware of their responsibility, especially in the context of dual-use, to use their skills in a reflective manner.

+

What is your perspective? 🤔

+
    +
  • Do you see differences in the overarching AI Ethics depending on the AI field?
    • +
  • What are your preferred AI Ethics Survey Papers?
    • +
  • Where do you consider AI Ethics in your daily R&D AI work?
    • +
  • What real issues could arise in ML models through the Wikipedia Chuck Norris fact (see image)?
    • +
+

I appreciate having a GenAI R&D team Comma Soft AG that is aware of the AI ethical implications of our work. +For more content, follow me on LinkedIn or reach out to me over PM ❤️

+

#AIethics #artificialintelligence #llm #genai #knowledgegraphs

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240610_will_we_reach_agi_and_if/index.html b/blog/240610_will_we_reach_agi_and_if/index.html new file mode 100644 index 0000000..0b27661 --- /dev/null +++ b/blog/240610_will_we_reach_agi_and_if/index.html @@ -0,0 +1,615 @@ + + + + + + Will we reach AGI, and if so, by transformers-based architectures? Share your perspective! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Will we reach AGI, and if so, by transformers-based architectures? Share your perspective!

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the potential of transformers-based architectures in achieving Artificial General Intelligence (AGI) and the ongoing debate surrounding it.

+

AGI Target

+

TL;DR ⏱️

+
    +
  • GenAI's impact on AGI discussions
    • +
  • Technical challenges with transformer-based architectures
    • +
  • Optimistic yet cautious approach at Comma Soft AG
    • +
+

Why GenAI boosted the discussions around AGI! 🤯

+
    +
  • Several institutions are claiming that our GenAI-based architectures are a key step on our way to AGI.
    • +
  • On the other hand, we are facing major technical issues with those systems that seem to not allow AGI by design: see stochastic parrots.
    • +
  • But still, some, like Ilya Sutskever, claim that transformer-architecture AIs can reach AGI-like capabilities as it does not only learn next token prediction, it learns a projection space for real-world understanding.
    • +
+ + +

We at Comma Soft AG are developing GenAI models being optimistic about the opportunities and seeing that GenAI is already helping in our tools and pipelines while being aware of all the current issues by design of these approaches (e.g., LLM hallucination) we are actively tackling.

+

Share your perspective in the comments 🤗 +For more content, follow me on LinkedIn ❤️

+

#LostInGenai #artificialintelligence #selectllm

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240617_do_we_need_another_genai_solution_how_why/index.html b/blog/240617_do_we_need_another_genai_solution_how_why/index.html new file mode 100644 index 0000000..b0d1be5 --- /dev/null +++ b/blog/240617_do_we_need_another_genai_solution_how_why/index.html @@ -0,0 +1,633 @@ + + + + + + Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on. + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on.

+ +
    +
    • +
  • blog
    • +
+ + +

An overview of the motivations and technical aspects behind developing our own GenAI solution, Alan, at Comma Soft AG.

+

Alan PDF

+

TL;DR ⏱️

+
    +
  • Diverse GenAI solutions exist
    • +
  • Unique motivations for developing our own tool
    • +
  • Technical advantages of our solution
    • +
  • Questions on custom development vs. wrappers
    • +
+

Current Landscape: 🏞️

+
    +
  • There are a variety of GenAI solutions out there
    • +
  • Many models from big tech companies work increasingly well for certain use cases.
    • +
  • And we have launched our own solution. Why?
    • +
+

Our technical perspective: 🤓

+
    +
  • Data-Flow-Control: We want to know what happens to our data.
    • +
  • Hosting: We host all our models and data on German servers.
    • +
  • On-Premise: We can run our entire tech stack on-premise without any internet connection
    • +
  • Our-own-Alan-Model: We optimize our models for dedicated use cases through further training (e.g., better language skills in GER)
    • +
  • Knowledge-Enrichment: We decide for ourselves whether we want to make new knowledge available through our novel training approach or through our optimized RAG approach (as you might know there are plenty of options).
    • +
  • Alignment: We have control over the alignment, i.e., to what extent the values and behaviors of the model match our expectations.
    • +
  • SOTA: We always remain state of the art in a fusion of existing open source base models and in-house developments.
    • +
  • Benchmarking: We use benchmarks and measures we trust.
    • +
+

With a great team at Comma Soft AG, it's enjoyable to develop such a helpful tool ❤️ In the appended PDF we show an extract of the full text story of our ALAN development journey 📖

+

Questions: 🤔

+
    +
  • How did you decide if you want to develop something yourself or use more kind of wrappers?
    • +
  • How important is the management of sensitive data for you and where are your servers located?
    • +
+

Literature: 📚

+ +

For more GenAI LLM content and R&D discussions, follow me on LinkedIn. If you want to try out Alan, reach out to me: http://alan.de

+

#artificialintelligence #llm #machinelearning #handson #genai

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240624_alternative_to_genai_creativity_watch_and_try/index.html b/blog/240624_alternative_to_genai_creativity_watch_and_try/index.html new file mode 100644 index 0000000..9830879 --- /dev/null +++ b/blog/240624_alternative_to_genai_creativity_watch_and_try/index.html @@ -0,0 +1,656 @@ + + + + + + Alternative to GenAI creativity? Watch and try out these fun Evolutionary Algorithms. Problem-solving without GenAI and SGD-based approaches explained! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Alternative to GenAI creativity? Watch and try out these fun Evolutionary Algorithms. Problem-solving without GenAI and SGD-based approaches explained!

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring Evolutionary Algorithms as an alternative to GenAI for problem-solving, using a fun 2D vehicle race example.

+

Evo Cars

+

TL;DR ⏱️

+
    +
  • There is hype around GenAI and LLMs
    • +
  • Evolutionary Algorithms (EAs) offer an alternative
    • +
  • A fun example of EAs using a 2D vehicle race
    • +
  • Steps involved in EAs explained
    • +
+

Background ☺️

+
    +
  • Within the field of AI is a GenAI and LLM Hype
    • +
  • These are not always the best approach to solve a problem!
    • +
  • There are interesting algorithms optimizing their behavior entirely differently: e.g., Evolutionary Algorithms (EA).
    • +
  • Today I want to show you a fun EA introduction based on a 2D vehicle race "problem"
    • +
+

Evolutionary algorithms (EAs) explained 👨🏼‍🏫

+

Problem 🏁:

+
    +
  • What is the best vehicle configuration to drive over a 2D surface?
    • +
+

Vehicle Configuration = Genome 🧬:

+
    +
  • Shape
    • +
  • Wheel size
    • +
  • Wheel position
    • +
  • Wheel weight
    • +
  • Chassis weight
    • +
+

Key EA Steps on the vehicle race example 🔢 :

+
    +
  1. Initialization: Generate a set of vehicles with random characteristics (genome) called population
    2. +
  2. Fitness Evaluation: Check how far they have got on the randomly generated surface.
    3. +
  3. Selection: Select a subset of the best vehicles from the population to reproduce and form the next generation, e.g., by tournament selection or rank selection.
    4. +
  4. Crossover (Recombination): Combine the genetic information of two or more selected vehicles to create new offspring. E.g., average two good car genomes or randomly select the information of each...
    5. +
  5. Mutation: Randomly modify the genetic information of some individuals in the population to introduce new variations and prevent convergence to a local optimum.
    6. +
  6. Replacement: Replace the least performing vehicles with new offspring generated through crossover and mutation.
    7. +
  7. Termination: Repeat until a stopping criterion is met: the needed distance, number of iterations, or convergence threshold.
    8. +
  8. Output: best vehicle, representing the near-optimal solution
    9. +
+

My Perspective 🤗:

+
    +
  • Be aware that there are a multitude of possible algorithms to use
    • +
  • We at Comma Soft AG do not trust any hype and select the appropriate algorithms for the respective problem
    • +
+

Credit ❤️

+
    +
  • To the development team of the interactive EA vehicle race
    • +
  • I'd also like to thank Hajira Jabeen and Jens Lehmann for their great support in my MT project where I worked on EA and KG for the creation of novel recipes
    • +
+ + +

Questions

+
    +
  • How would you solve this "race" problem? EA, GenAI, RL, ...?
    • +
  • Do you want to see more non-GenAI content?
    • +
  • Have you ever used EA to solve a problem?
    • +
+

Stay tuned by following me here on Linked ❤️

+

#LostInGenai #artificialintelligence #evolutionaryalgorithms

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240701_adjust_genai_responses_towards_more_ethical_behavior_possible/index.html b/blog/240701_adjust_genai_responses_towards_more_ethical_behavior_possible/index.html new file mode 100644 index 0000000..b8dc362 --- /dev/null +++ b/blog/240701_adjust_genai_responses_towards_more_ethical_behavior_possible/index.html @@ -0,0 +1,634 @@ + + + + + + Adjust GenAI responses towards more ethical behavior possible through system prompts!? Do you trust in such LLM-chat prepended pamphlets? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Adjust GenAI responses towards more ethical behavior possible through system prompts!? Do you trust in such LLM-chat prepended pamphlets?

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the potential and challenges of using system prompts to guide LLM behavior towards ethical outputs.

+

Ethical AI

+

TL;DR ⏱️

+
    +
  • GenAI chat interactions often include system prompts
    • +
  • System prompts aim to guide ethical LLM behavior
    • +
  • Challenges exist in ensuring compliance and formulation
    • +
  • Questions on designing and revealing system prompts
    • +
+

Background 🤓

+
    +
  • GenAI is often used via chat
    • +
  • We enter prompts in the Chat UI
    • +
  • Under the hood, the Chat programs usually add a system prompt
    • +
+

System-Prompt 💬

+
    +
  • This system prompt should control the behavior of the LLM output
    • +
  • The system prompt complements other behavioral optimizations such as alignment and other guardrails to improve ethical GenAI response
    • +
  • The system prompt can describe behavior such as identity or desired behavior
    • +
+

Problem 🥴

+
    +
  • There is no guarantee to what extent the system prompt will be followed
    • +
  • It is unclear what details should be included in the system prompt
    • +
  • It is not possible to prevent the model from revealing the system prompt
    • +
  • It is unclear how best to formulate the system prompt
    • +
+

Questions 🤔

+
    +
  • How do you design system prompts, and what are yours?
    • +
  • Do you think system prompts help to ensure the model's behavior?
    • +
  • Do you think chat UIs should not reveal the system prompt and how would you achieve this?
    • +
+

Further Reading 📖

+ +

We at Comma Soft AG develop GenAI solutions including optimizations of LLM responses through various components where also the design of system prompts plays a role.

+

#artificialintelligence #responsibleai #llm #alan #aiethics

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240708_genai_what_is_plagiarism_its_impact/index.html b/blog/240708_genai_what_is_plagiarism_its_impact/index.html new file mode 100644 index 0000000..385b887 --- /dev/null +++ b/blog/240708_genai_what_is_plagiarism_its_impact/index.html @@ -0,0 +1,635 @@ + + + + + + GenAI, what is plagiarism? Its Impact on Science. How should it be handled? What is your perspective? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

GenAI, what is plagiarism? Its Impact on Science. How should it be handled? What is your perspective?

+ +
    +
    • +
  • blog
    • +
+ + +

Discussing the implications of GenAI on scientific work and the thin line between acceptable use and plagiarism.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Use of GenAI in scientific work
    • +
  • Acceptable vs. debatable vs. critical usage
    • +
  • Questions and concerns about plagiarism
    • +
  • The pressure on researchers and students
    • +
  • Opportunities for better research
    • +
+

Current Situation 🏫

+

However, I see ChatGPT being used in many researchers' work to assist in the creation of (scientific) texts. Often it is also controlled or prohibited by the controlling institutions (schools, universities, paper-venues) to use such tools.

+

Sample GenAI-LLM usage I think is ok (IMHO) 🤗

+

I wonder how the following behavior in ChatGPT should be dealt with:

+
    +
  • You ask Chat GPT if it would correct commas and or capitalization in an existing text as Office Suites do
    • +
  • You ask for rewording from passive to active or better and more varied synonyms such as in Grammarly which is also "GenAI"
    • +
  • You can have texts translated to make quotes more easily accessible in the desired language like DeepL or Google Translate.
    • +
+

These are all uses of Chat-GPT that I can imagine in the context of scientific work that was previously possible without "ChatGPT" and, in my opinion, doable with other tools without having to speak of plagiarism.

+

Debatable usage 🤷🏼‍♂️

+
    +
  • You are generating the abstract for your finished paper while correcting and polishing it manually yourself afterwards.
    • +
+

More critical usage 🧐

+
    +
  • Write an entire text, paper/exercise/... simply based on a very refined prompt with LLM+RAG+Response-Verification and hand it in as your own work.
    • +
+

My Questions 👨🏼‍🎓

+

I just ask myself how I would have worked if the tools were there or how I would want to deal with these kinds of situations as a lecturer.

+
    +
  • What should be allowed?
    • +
  • What goes too far and falls under plagiarism?
    • +
  • Which use of (GenAI) tools needs to be regulated and documented and how finely tuned?
    • +
  • How do you think scientific work, reviews, and publications will change for better or for worse?
    • +
  • Does your institution at least clearly regulate how (GenAI-)Tool usage is possible
    • +
+

Current Problem 🫣

+

I see lots of pressure on students and research scientists to hand in their work or to publish papers. So they are tempted to use available tools. Not every GenAI will imply plagiarism but regulations are not clear as far as I have seen.

+

The Chance to Research 👨🏼‍🔬

+

We have the chance to use GenAI for better research: acquire good information over RAG and Vector Search, Translate texts into our mother tongue, and Rewrite in shorter or clearer language or even more accessible language.... Chat interact with papers you have questions to....

+

We at Comma Soft AG develop GenAI solutions and also actively using such tools to create great solutions while being aware and careful about possible plagiarism issues.

+

#genai #plagiarism #research #artificialintelligence

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240715_llms_big_vs_small_bigger_is_better_or_lets_not_waste_energy/index.html b/blog/240715_llms_big_vs_small_bigger_is_better_or_lets_not_waste_energy/index.html new file mode 100644 index 0000000..6cf8ab0 --- /dev/null +++ b/blog/240715_llms_big_vs_small_bigger_is_better_or_lets_not_waste_energy/index.html @@ -0,0 +1,637 @@ + + + + + + LLMs - Big vs Small. Bigger is Better!? OR Let's not waste energy!? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

LLMs - Big vs Small. Bigger is Better!? OR Let's not waste energy!?

+ +
    +
    • +
  • blog
    • +
+ + +

The AI community is abuzz with debates over the efficacy of large versus small language models. Both have their own merits and limitations.

+

Model Size

+

TL;DR ⏱️

+
    +
  • AI community debates model sizes
    • +
  • Massive models vs. smaller, efficient models
    • +
  • Insights and future predictions
    • +
  • Links to further reading
    • +
+

✨ The AI community is buzzing with discussions about model sizes:

+
    +
  • Massive models like Mistral (8x22B), LLAMA3 (400B), and Grok (314B) are turning heads.
    • +
  • Smaller yet mighty models like Phi 3, LLAMA3 (8B), and Nemo (12B) are proving their worth.
    • +
+

🔍 Key insights:

+
    +
  • Smaller models can compete by focusing on smarter training and pre-tuning methods.
    • +
  • Benchmarks are helpful but not flawless in measuring a model's value.
    • +
  • The mystery remains: why might a model with 61 transformer layers outperform one with 60?
    • +
  • Balancing model architecture is crucial due to the increasing complexity as models scale up.
    • +
+

🔮 Future predictions:

+
    +
  • Will we see a standard model size emerge, or will there be a variety for different model size clusters for different platforms (mobile, single GPU, multi-GPU clusters)?
    • +
+ + +

🌐 What is your experience?

+
    +
  • Which model size do you find most practical for real-world problems?
    • +
+

📊 I'm keen to learn from you:

+
    +
  • What size models are you working with?
    • +
  • Share your preferences in this survey and let's discuss the optimal balance in model scaling.
    • +
  • What do you think how big is the gpt4o turbo?
    • +
  • Join the conversation and let's navigate the evolving landscape of machine learning together ❤️
    • +
+

At Alan (by Comma Soft AG), we recognize the need for variety: +We provide models of various sizes to align with your unique requirements for capability and speed.

+

#MachineLearning #AI #ModelSize #Innovation #GenAI #SustainableAI

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240722_these_results_give_me_hope_for_sustainable_ai/index.html b/blog/240722_these_results_give_me_hope_for_sustainable_ai/index.html new file mode 100644 index 0000000..d72fb02 --- /dev/null +++ b/blog/240722_these_results_give_me_hope_for_sustainable_ai/index.html @@ -0,0 +1,608 @@ + + + + + + These results give me hope for sustainable AI 🌱 + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

These results give me hope for sustainable AI 🌱

+ +
    +
    • +
  • blog
    • +
+ + +

I'm impressed by some of the recent advances in the field of "small" open-weight Language Models (LLMs).

+

Sustainable AI

+

TL;DR ⏱️

+
    +
  • Increased documentation supports reproducibility
    • +
  • Data quality improves model performance
    • +
  • Model distillation reduces hardware needs
    • +
+

More Documentation 📚

+

They're accompanied by increased documentation, as seen with efforts from Apple and Meta, which support reproducibility and reduce wasted effort and energy on less promising pre-training and fine-tuning iterations.

+

Data Quality as Chance 🔎

+

They demonstrate that a focus on data quality can improve model performance, as evidenced by Phi-3. However, it's also clear that the total number of tokens contributes to improvements, as seen with LLAMA3(.1).

+

Model Distillation 👩🏽‍🔬

+

Model Distillation has been effective in reducing total hardware requirements and inference time, which saves energy and resources. It will be interesting to see how quickly distilled models like Gemma2 can outperform 6-month-old state-of-the-art models like LLAMA2.

+

I appreciate the efforts to decrease hardware and energy demands while still providing helpful model responses.

+

#artificialintelligence #genai #sustainableai #llm

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240729_what_is_the_perfect_approach_to_adjust_an_llm_to_your_genai_use_case/index.html b/blog/240729_what_is_the_perfect_approach_to_adjust_an_llm_to_your_genai_use_case/index.html new file mode 100644 index 0000000..21da54d --- /dev/null +++ b/blog/240729_what_is_the_perfect_approach_to_adjust_an_llm_to_your_genai_use_case/index.html @@ -0,0 +1,624 @@ + + + + + + What is the perfect approach to adjust an LLM to your GenAI use case? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

What is the perfect approach to adjust an LLM to your GenAI use case?

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring various methods to customize LLMs for specific GenAI use cases, ranging from simple to complex approaches.

+

Model Training

+

TL;DR ⏱️

+
    +
  • Various ways to customize LLMs for specific use cases
    • +
  • Approaches vary in difficulty and complexity
    • +
  • Pros and cons of different methods
    • +
  • More dimensions to improve GenAI use cases
    • +
+

Key Considerations:

+

Prompt Engineering & In-Context Learning: Manually enriching the prompt with information to guide the model’s prediction into desired behavior.

+

Retrieval-Augmented Generation: Automatically retrieving context that is appended to the prompt on the fly after being retrieved through vector database similarity search and reranking.

+

Parameter-Efficient Finetuning: Training only a subset of the model's parameters while keeping others unchanged.

+

Full Parameter Finetuning: Training all parameters.

+

These approaches have pros and cons in terms of (and not limited to):

+
    +
  • Catastrophic Forgetting
    • +
  • Memory Load
    • +
  • Maximum Context-Size limits
    • +
  • Hallucinations
    • +
  • Response Time
    • +
+

There are also more approaches and dimensions to improve GenAI Use Cases like:

+
    +
  • Guardrails
    • +
  • Structured output +and many more.
    • +
+

What are you working on, and which field are you interested in to see how we are working on it?

+

We at Comma Soft AG implement these and more approaches into our own Product Alan.de and support also other GenAI Use Cases through these approaches.

+

Reach out to me 🤗 and follow me for more content ❤️

+

#artificialintelligence #genai #llm

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240830_combining_the_hugging_face_model_platform_and_knowledge/index.html b/blog/240830_combining_the_hugging_face_model_platform_and_knowledge/index.html new file mode 100644 index 0000000..bf7f873 --- /dev/null +++ b/blog/240830_combining_the_hugging_face_model_platform_and_knowledge/index.html @@ -0,0 +1,602 @@ + + + + + + Combining the Hugging Face Model Platform and Knowledge Graph trend analysis over time could improve GenAI research and reduce waste of energy! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Combining the Hugging Face Model Platform and Knowledge Graph trend analysis over time could improve GenAI research and reduce waste of energy!

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the potential of leveraging knowledge graphs to analyze trends in evolving models for better GenAI research and efficiency.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Leveraging knowledge graphs for GenAI trends
    • +
  • Identifying high-performing models and best practices
    • +
  • Potential for a crowd-sourced GenAI cookbook
    • +
+

Traversing the possible huge knowledge graph of evolving models... It would be interesting to see trends within this graph, such as which families and architectures are trending 📈. I'd love to explore over time what approaches for fine-tuning, datasets, or even the number of transformer layers/heads, etc. create high-performing and efficient models 🌱. So we could create a crowd-sourced best practice GenAI cookbook ❤️👩🏽‍🍳. Please let me know if you are already working on it Thomas Wolf 🤗

+

#LostInGenai #artificialintelligence #huggingface #knowledgegraphs #genairesearch

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240903_sustainable_air_gapped_on_prem_llm_solution/index.html b/blog/240903_sustainable_air_gapped_on_prem_llm_solution/index.html new file mode 100644 index 0000000..07ff77b --- /dev/null +++ b/blog/240903_sustainable_air_gapped_on_prem_llm_solution/index.html @@ -0,0 +1,628 @@ + + + + + + Sustainable Air-Gapped On-Prem LLM Solution! How can we make GenAI available on almost any hardware, and how is it also available as a portable demo on our Alan Notebook + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Sustainable Air-Gapped On-Prem LLM Solution! How can we make GenAI available on almost any hardware, and how is it also available as a portable demo on our Alan Notebook

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the development of a full-stack GenAI LLM solution that can run on a variety of hardware configurations, including a portable demo setup.

+

Alan Notebook

+

TL;DR ⏱️

+
    +
  • Developing Alan, a full-stack GenAI LLM solution
    • +
  • Hosted on German hyperscaler infrastructure
    • +
  • Offers a smaller version, Alan-S-LLM
    • +
  • Portable demo available on Alan Notebook
    • +
+

What We're Working on When We Talk About Alan R&D 👨🏼‍💻

+
    +
  • In a recent post, I introduced how we're developing Alan, a full-stack GenAI LLM solution.
    • +
  • We host our solution within German hyperscaler infrastructure to deal with the requirements of multiple customer tenants and our large language models, including retrieval augmented generation pipelines.
    • +
  • The requirements of our strongest Alan LLM require current top-notch Nvidia GPUs (Ampere+, 80GB VRAM), but we also offer a smaller Alan-S-LLM, which still has tremendous capabilities with fewer hardware requirements.
    • +
+

How to Shrink LLMs 🤖

+
    +
  • Models are smaller in dimensions like the number of transformer layers, heads, hidden dimensions, and other hyperparameters.
    • +
  • Current smaller GenAI LLMs can be designed by model distillation and model pruning, which try to keep model quality high while reducing the number of parameters.
    • +
  • The reduced number of parameters reduces VRAM requirements. Fewer parameters, especially fewer transformer layers, increase the throughput and inference performance as well.
    • +
  • The reduction of bits used to represent each parameter of the LLM reduces the required total GPU VRAM.
    • +
+

Our Alan Demo Notebook 💻

+
    +
  • To demonstrate that our entire tech stack is capable of running entirely air-gapped and to showcase that we're truly capable of showing this on even a portable system, we developed the Alan-Notebook.
    • +
  • This notebook uses the entire tech stack, which includes all the components that offer Multi-GPU Cluster setups, handling of users, RAG pipelines, and, of course, LLM text inference.
    • +
  • The model behind is our fastest and most efficient Alan-S-Model. The notebook has limited hardware capabilities, especially within the GPU (16GB, Nvidia), but can still run the entire tech stack.
    • +
+

Further Reading:

+ +

If you're interested in how we develop an entire scalable GenAI solution and you want to see some details into our R&D, follow me on LinkedIn and reach out to us. Thanks to my wonderful teammates Dr. Laura Maaßen and Lars Flöer, who made this Alan Notebook possible, and to the entire Alan Development Team Comma Soft AG for supporting this great project and product.

+

#genai #onprem #llm #alan

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240910_for_more_sustainability_transparency_in_genai_share/index.html b/blog/240910_for_more_sustainability_transparency_in_genai_share/index.html new file mode 100644 index 0000000..f577acc --- /dev/null +++ b/blog/240910_for_more_sustainability_transparency_in_genai_share/index.html @@ -0,0 +1,617 @@ + + + + + + For more sustainability transparency in GenAI! Share your knowledge and reduce energy waste! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

For more sustainability transparency in GenAI! Share your knowledge and reduce energy waste!

+ +
    +
    • +
  • blog
    • +
+ + +

Emphasizing the importance of transparency and shared knowledge to enhance sustainability in GenAI.

+

Image 1

+

TL;DR ⏱️

+
    +
  • GenAI involves very large models and significant training efforts
    • +
  • Transparency can help share emissions and reduce energy waste
    • +
  • Open source models can optimize future development
    • +
+

GenAI and Sustainability 🌱

+
    +
  • GenAI is implemented through very large models.
    • +
  • The training can be substantial, such as the 15T tokens of LLAMA 3.1.
    • +
  • However, these trainings represent only the final training and not all the attempts that led to the final model.
    • +
+

How Transparency helps 📚

+
    +
  • If the models are open or, in the best case, open source, many users and use cases can share the resulting emissions.
    • +
  • It would also be interesting to see how often models are used for inference to compare the relative share of emissions.
    • +
  • Open source would be important to share learnings for the next generation of models and thus reduce emissions by reusing hyperparameter optimizations. +Please share your findings and let's use our resources and energy wisely.
    • +
+

Our Paper 📄

+

In our paper “Ethical and Sustainability Considerations for Knowledge Graph based Machine Learning,” we highlight which sustainability and ethical optimizations are possible when working with hardware-intensive Artificial Intelligence approaches. Link: https://lnkd.in/eJjATkTQ

+

What are you doing to bring AI, ethics, and sustainability together? ❤️

+

#artificialintelligence #genai #llm #sustainability #aiethics

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240917_todays_research_proposal_how_to_achieve_real/index.html b/blog/240917_todays_research_proposal_how_to_achieve_real/index.html new file mode 100644 index 0000000..2c5662e --- /dev/null +++ b/blog/240917_todays_research_proposal_how_to_achieve_real/index.html @@ -0,0 +1,629 @@ + + + + + + Today's Research Proposal - How to achieve "real" thinking and reasoning in GenAI, rather than just relying on a silent Chain of Thought, as seen in ReflectionAI or possibly GPT-o1? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Today's Research Proposal - How to achieve "real" thinking and reasoning in GenAI, rather than just relying on a silent Chain of Thought, as seen in ReflectionAI or possibly GPT-o1?

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the potential for achieving true reasoning and thinking in Generative AI models beyond the current Chain of Thought methodologies.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Current state of reasoning in models
    • +
  • Possibilities for transformers to learn to think
    • +
  • Customization ideas for achieving true reasoning
    • +
  • Open questions and discussion points
    • +
+

Currently, reasoning more through silent the CoT 😥

+
    +
  • There are currently models that claim to possess reasoning capabilities.
    • +
  • However, this is more about the Chain of Thought and the use of special tokens.
    • +
  • These enable the model to generate more text, thereby increasing the stability of its answers.
    • +
  • This process of "reasoning" is not transparently/barely shown.
    • +
  • I believe this is less about reasoning or thinking and more about statistical stability.
    • +
  • Moreover, it is not possible to observe the reasoning process, and it is often difficult to identify when a model has gone astray.
    • +
+

How could transformers learn to think? 🤯

+
    +
  • Why can I still imagine that it is possible to learn to think or reason with the transformer architecture?
    • +
  • Due to the transformer architecture, which includes tokenization, attention, and multi-layer perceptron elements, as well as the universal function approximation hypothesis, the following can be imagined: +
      +
    1. In early layers, the network learns something akin to named entity recognition.
      2. +
    2. Later layers and embedding dimensions create structures that learn propositional logic and structures similar to knowledge graphs or their embeddings.
      3. +
    3. Finally, based on this, resonating and reflecting completeness of output logic components could be achieved.
      4. +
    +
    • +
+

What else would I customize? 👨🏼‍💻 (IMHO)

+

I would find it interesting to combine this with thought-diffusion models, making hierarchical planning possible. I have several more ideas..., reach out to me if you like to start a discussion ❤️

+

Questions:

+

Are you aware of any approaches or research projects that attempt this? +What are your thoughts on that, how would you build "AGI" it?

+

I had great idea exchanges on this with colleagues from Comma Soft AG, Lamarr-Institut & Smart Data Analytics.

+

Let's not waste money and time on believing in marketing claims and rebranding of the word "reasoning", but let's start to think of how it could actually be achieved 🤗

+

#AGI #GenAI #artificialintelligence #research

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/240924_where_science_meets_innovation_my_personal/index.html b/blog/240924_where_science_meets_innovation_my_personal/index.html new file mode 100644 index 0000000..be19e93 --- /dev/null +++ b/blog/240924_where_science_meets_innovation_my_personal/index.html @@ -0,0 +1,615 @@ + + + + + + Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions? + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions?

+ +
    +
    • +
  • blog
    • +
+ + +

Highlights and open questions from the Petersberger Gespräche (PG) 2024, covering AI, energy transition, chip technologies, and more.

+

PG 2024 Highlights

+

TL;DR ⏱️

+
    +
  • AI and consciousness discussions
    • +
  • Energy transition and regulatory challenges
    • +
  • Distributed chip technologies in Europe
    • +
  • Generative AI in media
    • +
  • Metaverse applications beyond gaming
    • +
+

🧠 AI and Consciousness 🧠

+

Joscha Bach from Liquid AI presented on how advancements in AI could potentially redefine our understanding of consciousness. My open question for him is: How does he intend to reliably measure the achievement of AI consciousness in his efforts?

+

👩🏻‍🔬 Scientific Chemical Hands on how to fix energy transition 👩🏻‍🔬

+

As chairman of Alexander von Humboldt Foundation, Robert Schlögl delivered an impressively passionate presentation on energy tech for the climate transition and the phasing out of fossil fuels. Besides scientific derivations, he also highlighted significant regulatory issues that hinder successful implementation. I wonder: How these regulations came about, why they are justified if they are (potentially) not scientifically tenable, and how they can now be resolved?

+

🇪🇺 Efficient Distributed Chip Technologies from the heart of Europe 🇪🇺

+

Christian Mayr from Technische Universität Dresden discussed the potential and developments possible in Dresden. He mentioned the use of clustered chip technologies to operate LLMs. One of my open questions is: How can LLMs be distributed across tens of thousands of chips while still achieving acceptable inferential latencies?

+

📰 GenAI in Media, Opportunities and Risks 📰

+

In further discussions with Sibylle Anderl from ZEIT Verlagsgruppe, we explored the potentials and risks of Generative AI in text, image, and video generation, and how reducing anonymity could potentially restore credibility. My open question: Whether there is a reliable way to recognize these outputs, given that word frequency alone might not be a proof?

+

🕹️ Metaverse, not only a Playground? 🕹️

+

Nico Michels from Siemens presented the potentials of the Metaverse and digital twins. Q: I'd love to see how such models can help even small municipalities with climate adaptation models, regional impact assessments, and improvement options?

+

The open questions highlight the stimulating topics the diverse guests who aim to drive positive change with a progressive mindset. I would like to thank Comma Soft AG and everyone involved for organizing this fantastic event, especially Stephan Huthmacher, whose heartfelt dedication makes this recurring, inspiring event possible. ❤️

+

Share your thoughts and impressions, I'll share the streams 🤗

+

#artificialintelligence #genai #sustainability #petersbergergespräche

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/241001_stop_adding_languages_to_llms_the_potential/index.html b/blog/241001_stop_adding_languages_to_llms_the_potential/index.html new file mode 100644 index 0000000..5ff2fc2 --- /dev/null +++ b/blog/241001_stop_adding_languages_to_llms_the_potential/index.html @@ -0,0 +1,627 @@ + + + + + + Stop adding Languages to LLMs! The Potential Drawbacks of Training Multilingual Large Language Models (LLMs) for Performance and Sustainability! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Stop adding Languages to LLMs! The Potential Drawbacks of Training Multilingual Large Language Models (LLMs) for Performance and Sustainability!

+ +
    +
    • +
  • blog
    • +
+ + +

Exploring the downsides of creating multilingual LLMs and their impact on performance and resource utilization.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Challenges of building multilingual LLMs
    • +
  • Inefficiencies in token usage and context length
    • +
  • Increased hardware costs and reduced token training
    • +
  • Weighing multilingual models against language-specific models
    • +
+

Building a Multi-Lang-LLM 🛠️🐣

+
    +
  • When pretraining LLMs, one of the key decisions is which data to include.
    • +
  • This choice also influences the selection of the tokenizer, which optimizes the number of tokens for the texts used.
    • +
  • As a result, different characters and character sequences are mapped in the tokenizers for each language.
    • +
+

When you finally use such a LLM in only a subset of available languages you face following problems 🇪🇺🤖

+
    +
  • Inefficient token usage: If the model is only used for one or two languages, many tokens may be rarely or never needed, leading to shorter token sequences in the required language.
    • +
  • Limited context length: LLMs have a limited context length, measured in tokens, which can result in more expensive inference as the model scales linearly to quadratically with prompt length.
    • +
  • Increased hardware costs: This can lead to higher hardware costs and omissions.
    • +
  • Reduced relevant token training: With a multilingual model, fewer relevant tokens and token sequences may have been seen and trained in the required languages.
    • +
+

The Trade-Off: Multilingual Models vs. Language-Specific Models 💰📊

+
    +
  • We need to weigh the benefits of multilingual models against the potential drawbacks and decide whether to prioritize language coverage or risk wasting resources.
    • +
  • This is particularly important when dealing with languages that are not closely related, such as those with different character sets.
    • +
+

Your Opinion 🤗

+
    +
  • What do you think?
    • +
  • Have you ever chosen a model especially with reduced number of languages outside English?
    • +
  • Some more details about Multi-Lang-GenAI can be found here: https://lnkd.in/edgPsdKz
    • +
+

For more content, follow me or reach out to me over DM ❤️

+

#artificialintelligence #genai #llm #languages

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/241008_choosing_the_right_llm_for_your_needs_key_considerations/index.html b/blog/241008_choosing_the_right_llm_for_your_needs_key_considerations/index.html new file mode 100644 index 0000000..29905f7 --- /dev/null +++ b/blog/241008_choosing_the_right_llm_for_your_needs_key_considerations/index.html @@ -0,0 +1,620 @@ + + + + + + Choosing the Right LLM for Your Needs - Key Considerations + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

Choosing the Right LLM for Your Needs - Key Considerations

+ +
    +
    • +
  • blog
    • +
+ + +

Consider the key factors when selecting a Large Language Model (LLM) for your project.

+

Image 1

+

TL;DR ⏱️

+
    +
  • Benchmark Performance
    • +
  • License
    • +
  • Model Size
    • +
  • Alignment
    • +
+

Key Considerations:

+
    +
  • 📊 Benchmark Performance: Consider the model's performance on relevant benchmarks known from Open LLM Leaderboard and especially Arena Elo. https://lnkd.in/eSkeAUV7
    • +
  • 📜 License: Ensure the license aligns with your project's requirements and complies with any regulatory restrictions. https://lnkd.in/e8V-eMCh
    • +
  • 📊 Model Size: Consider the trade-off between model size and performance for your specific use case. https://lnkd.in/egZt7BmJ
    • +
  • 🔄 Alignment: Evaluate the alignment process and whether it's transparent, as this can impact the model's performance and reliability. https://lnkd.in/eViiEyqp
    • +
  • 🔍 Transparency of Training: Look for models with transparent training data and methods to ensure you understand how the model was trained.
    • +
  • 💸 Inference Costs: Assess the model's inference costs and consider the trade-off between performance and cost. https://lnkd.in/etSajZZc
    • +
  • 📚 Context Size: Consider the model's context size and whether it's suitable for your specific use case.
    • +
  • 🤝 Compatibility: Evaluate whether the model is compatible with SOTA libraries like transformers and whether it's easily integrable into your workflow.
    • +
  • 📈 Scaling Efficiency: Assess the model's scaling efficiency or e.g. it has full quadratic complexity with more input tokens.
    • +
  • 🌎 Multilingualism: Evaluate the model's multilingual capabilities it's effect for your specific use case. https://lnkd.in/eeVsG99M
    • +
+

What Do You Think?

+
    +
  • What is your importance-order of LLM features you look at? Which criteria do you miss in this list or which should be ranked higher?
    • +
  • Within our Projects Comma Soft AG these are some of the major criteria we look at when we are selecting GenAI models like LLMs.
    • +
  • If you like to see more of best practice content, follow me, share your thoughts, and leave me a like ❤️
    • +
+

#LostInGenai #artificialintelligence #selectllm

+ + + + +
+ + + + + + + diff --git a/blog/241016_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/index.html b/blog/241016_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/index.html new file mode 100644 index 0000000..72605bc --- /dev/null +++ b/blog/241016_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/index.html @@ -0,0 +1,605 @@ + + + + + + We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity!

+ +
    +
    • +
  • blog
    • +
+ + +

We will see more and more precise adjustment options in GenAI Tooling

+

Image 1

+

TL;DR ⏱️

+
    +
  • GenAI often lacks precise Adjuments
    • +
  • Tools and Models start CLosing the Gap
    • +
  • Major Waste of Time and Resources through inefficient Regenerates
    • +
+

Control and hallucinations in generated videos:

+

Generative AI that looks nice, I’m wondering if the camera turns multiple laps around the guy in the forest, if the color of the jacket on the back would stay the same without having it explicitly described. What do you think, what would happen? 📹🧥

+

Importance of GenAI control:

+

I think overall the opportunity to control generations by precise prompts or other interfaces to further specify intermediate results will be insanely important if these technologies should be adapted in a high efficient outcome-oriented business. We cannot waste time for prompts and regenerates! Cause of time and energy 🌱⏰

+

#artificialintelligence #genai

+

LinkedIn Post

+ + + + +
+ + + + + + + diff --git a/blog/241025_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/index.html b/blog/241025_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/index.html new file mode 100644 index 0000000..7f8c5ea --- /dev/null +++ b/blog/241025_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/index.html @@ -0,0 +1,629 @@ + + + + + + A GenAI Solution/Tweak to Rescue ARD tagesschau webpage?! + + + + + + + + +
+
Carsten Felix Draschner, PhD
+ +
+ +
+ + +

A GenAI Solution/Tweak to Rescue ARD tagesschau webpage?!

+ +
    +
    • +
  • blog
    • +
+ + +

Brainstorming how old regulations meet new GenAI opportunities.

+

Image 1

+

TL;DR 📝

+
    +
  • Debate over new regulations restricting text content of public broadcasters online.
    • +
  • Reform could impact news portals like tagesschau.de by limiting text content to only that which accompanies broadcasts.
    • +
  • GenAI could potentially auto-generate videos from text content to comply with regulations.
    • +
  • Questions about the impact and regulation of GenAI in public broadcasting contexts.
    • +
+

Disclaimer

+
    +
  • I am wondering if my proposed solution would contradict the possible upcoming regulations (see below) 🤔.
    • +
  • I might be biased as tagesschau.de is for me one good source of news with a mix of text and video while being free of ads 📰(IMHO).
    • +
  • I am writing this as only one example where old regulations might clash with or ignore recent GenAI developments.
    • +
+

Current Debate:

+
    +
  • More or less every household needs to pay ~18€ monthly for public broadcast 💶.
    • +
  • State premiers are discussing a reform that would significantly restrict the text content of public broadcasters like ARD and ZDF online 📄.
    • +
  • The draft reform mandates that only texts accompanying broadcasts are permitted, meaning text content can only be published if it has been previously covered in a TV broadcast 📺.
    • +
  • This change would greatly impact the speed and diversity of reporting, particularly for news portals like tagesschau.de, they say 🚫.
    • +
  • Private publishers, viewing this as a competitive advantage, support the reform, while public broadcaster representatives warn of negative effects on media diversity and information provision 🏢.
    • +
+

My GenAI Perspective:

+
    +
  • What would happen if these platforms autogenerate videos on some digital video stream platform based on the text content they propose? 🤖.
    • +
  • We have seen such approaches of text-to-video generation or speech synthesis 📝.
    • +
  • Maybe with a digital clone of some professional or artificial speaker (who needs to allow this for sure and should get paid for giving away the digital twin) 🗣️.
    • +
  • Currently we face several conflicts of old media and GenAI but this is more between public broadcast and private publishers ⚖️.
    • +
+

Questions:

+
    +
  • How do you see it? 🤷.
    • +
  • Do you think GenAI content would solve their issue? 🤔.
    • +
  • How is it regulated within your country if public broadcast can write articles? 🌍.
    • +
+

Link to 🇩🇪 Tagesschau article: https://lnkd.in/dkx_RZun

+

#Rundfunkbeitrag #genai #gez #presse #artificialintelligence

+ + + + +
+ + + + + + + diff --git a/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/index.html b/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/index.html index 593bd50..a13b87e 100644 --- a/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/index.html +++ b/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/index.html @@ -569,7 +569,7 @@

The Uncanny Valley Phenomenon in GenAI Face Synthesis

@@ -606,7 +606,7 @@

What Do You Think?

#genai #deeptech #artificialintelligence #uncannyvalley

- - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • We're Dealing with the Butterfly Effect in LLM Creation Pipelines 🦋🤖 - +

    When we develop novel LLMs, AI-Agents and GenAI Pipelines for Alan.de or within our various other GenAI projects, we're continuously learning about the Butterfly Effect in GenAI creation pipelines and how to mitigate this problem. 👨🏼‍💻 🤯 Every change can have a huge impact on the entire pipeline that creates and executes models

    Image 1

    @@ -433,27 +433,12 @@

    Blog Posts

  • Strategies to mitigate the Butterfly Effect are essential for optimal performance.
    • - (Read More)  (Original-Websource) - - -
  • - We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! - -

    We will see more and more precise adjustment options in GenAI Tooling

    -

    Image 1

    -

    TL;DR ⏱️

    - -
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • New Official OSI "Open Source AI definition" attacks Meta's LLAMA as "Open Washing" - +

    We getting closer what should be considered being open source in GenAI and not only open weight.

    Image 1

    TL;DR 📝

    @@ -464,12 +449,12 @@

    Blog Posts

  • "Open Washing" Problem of Meta and others?!
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • The Uncanny Valley Phenomenon in GenAI Face Synthesis - +

    GenAI hits Uncanny Valley Problems as we have seen in Animation and Computer Games

    Image 1

    TL;DR 📝

    @@ -485,7 +470,7 @@

    Blog Posts

  • A GenAI Solution/Tweak to Rescue ARD tagesschau webpage?! - +

    Brainstorming how old regulations meet new GenAI opportunities.

    Image 1

    TL;DR 📝

    @@ -496,12 +481,27 @@

    Blog Posts

  • Questions about the impact and regulation of GenAI in public broadcasting contexts.
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource) + + +
  • + We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! + +

    We will see more and more precise adjustment options in GenAI Tooling

    +

    Image 1

    +

    TL;DR ⏱️

    + +
    + (Read More)  (Original-Websource)
  • Choosing the Right LLM for Your Needs - Key Considerations - +

    Consider the key factors when selecting a Large Language Model (LLM) for your project.

    Image 1

    TL;DR ⏱️

    @@ -512,12 +512,12 @@

    Blog Posts

  • Alignment
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Stop adding Languages to LLMs! The Potential Drawbacks of Training Multilingual Large Language Models (LLMs) for Performance and Sustainability! - +

    Exploring the downsides of creating multilingual LLMs and their impact on performance and resource utilization.

    Image 1

    TL;DR ⏱️

    @@ -528,12 +528,43 @@

    Blog Posts

  • Weighing multilingual models against language-specific models
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • - Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions? + Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on. +

    An overview of the motivations and technical aspects behind developing our own GenAI solution, Alan, at Comma Soft AG.

    +

    Alan PDF

    +

    TL;DR ⏱️

    + +
    + (Read More)  (Original-Websource) +
    • + +
  • + DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes + +

    Interesting findings on DALLE's content filtering mechanisms.

    +

    Image 1

    +

    TL;DR ⏱️

    + +
    + (Read More)  (Original-Websource) +
    • + +
  • + Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions? +

    Highlights and open questions from the Petersberger Gespräche (PG) 2024, covering AI, energy transition, chip technologies, and more.

    PG 2024 Highlights

    TL;DR ⏱️

    @@ -545,12 +576,12 @@

    Blog Posts

  • Metaverse applications beyond gaming
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Today's Research Proposal - How to achieve "real" thinking and reasoning in GenAI, rather than just relying on a silent Chain of Thought, as seen in ReflectionAI or possibly GPT-o1? - +

    Exploring the potential for achieving true reasoning and thinking in Generative AI models beyond the current Chain of Thought methodologies.

    Image 1

    TL;DR ⏱️

    @@ -561,12 +592,12 @@

    Blog Posts

  • Open questions and discussion points
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • For more sustainability transparency in GenAI! Share your knowledge and reduce energy waste! - +

    Emphasizing the importance of transparency and shared knowledge to enhance sustainability in GenAI.

    Image 1

    TL;DR ⏱️

    @@ -576,12 +607,12 @@

    Blog Posts

  • Open source models can optimize future development
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Sustainable Air-Gapped On-Prem LLM Solution! How can we make GenAI available on almost any hardware, and how is it also available as a portable demo on our Alan Notebook - +

    Exploring the development of a full-stack GenAI LLM solution that can run on a variety of hardware configurations, including a portable demo setup.

    Alan Notebook

    TL;DR ⏱️

    @@ -592,12 +623,12 @@

    Blog Posts

  • Portable demo available on Alan Notebook
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Combining the Hugging Face Model Platform and Knowledge Graph trend analysis over time could improve GenAI research and reduce waste of energy! - +

    Exploring the potential of leveraging knowledge graphs to analyze trends in evolving models for better GenAI research and efficiency.

    Image 1

    TL;DR ⏱️

    @@ -607,12 +638,12 @@

    Blog Posts

  • Potential for a crowd-sourced GenAI cookbook
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • What is the perfect approach to adjust an LLM to your GenAI use case? - +

    Exploring various methods to customize LLMs for specific GenAI use cases, ranging from simple to complex approaches.

    Model Training

    TL;DR ⏱️

    @@ -623,12 +654,12 @@

    Blog Posts

  • More dimensions to improve GenAI use cases
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • These results give me hope for sustainable AI 🌱 - +

    I'm impressed by some of the recent advances in the field of "small" open-weight Language Models (LLMs).

    Sustainable AI

    TL;DR ⏱️

    @@ -638,12 +669,12 @@

    Blog Posts

  • Model distillation reduces hardware needs
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • LLMs - Big vs Small. Bigger is Better!? OR Let's not waste energy!? - +

    The AI community is abuzz with debates over the efficacy of large versus small language models. Both have their own merits and limitations.

    Model Size

    TL;DR ⏱️

    @@ -654,12 +685,12 @@

    Blog Posts

  • Links to further reading
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • GenAI, what is plagiarism? Its Impact on Science. How should it be handled? What is your perspective? - +

    Discussing the implications of GenAI on scientific work and the thin line between acceptable use and plagiarism.

    Image 1

    TL;DR ⏱️

    @@ -671,12 +702,12 @@

    Blog Posts

  • Opportunities for better research
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Adjust GenAI responses towards more ethical behavior possible through system prompts!? Do you trust in such LLM-chat prepended pamphlets? - +

    Exploring the potential and challenges of using system prompts to guide LLM behavior towards ethical outputs.

    Ethical AI

    TL;DR ⏱️

    @@ -687,12 +718,12 @@

    Blog Posts

  • Questions on designing and revealing system prompts
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Alternative to GenAI creativity? Watch and try out these fun Evolutionary Algorithms. Problem-solving without GenAI and SGD-based approaches explained! - +

    Exploring Evolutionary Algorithms as an alternative to GenAI for problem-solving, using a fun 2D vehicle race example.

    Evo Cars

    TL;DR ⏱️

    @@ -703,28 +734,12 @@

    Blog Posts

  • Steps involved in EAs explained
    • - (Read More)  (Original-Websource) - - -
  • - Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on. - -

    An overview of the motivations and technical aspects behind developing our own GenAI solution, Alan, at Comma Soft AG.

    -

    Alan PDF

    -

    TL;DR ⏱️

    - -
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Will we reach AGI, and if so, by transformers-based architectures? Share your perspective! - +

    Exploring the potential of transformers-based architectures in achieving Artificial General Intelligence (AGI) and the ongoing debate surrounding it.

    AGI Target

    TL;DR ⏱️

    @@ -734,12 +749,12 @@

    Blog Posts

  • Optimistic yet cautious approach at Comma Soft AG
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Do you differentiate AI Ethics principles between AI/ML fields like GenAI/LLMs or Knowledge Graph-based ML? How do we deal with so-called facts on the internet as training data? - +

    Exploring the nuances of AI ethics across different AI/ML fields and handling internet-based training data responsibly.

    Ethics in AI

    TL;DR ⏱️

    @@ -750,12 +765,12 @@

    Blog Posts

  • Questions to ponder on AI ethics practices
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • What expectations do you have regarding the values and norms of your GenAI chat assistants? Highly sensitive topic in the LLM space! My take... - +

    Exploring the ethical considerations and expectations surrounding the values and norms embedded in GenAI chat assistants.

    Image 1

    TL;DR ⏱️

    @@ -766,12 +781,12 @@

    Blog Posts

  • Need for a holistic view on model behavior
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Be careful when you speak of Open (Source) GenAI. Why OpenAI and Meta (shouldn't) use the word Open within their GenAI efforts? - +

    Examining the implications of using the term "Open" in the context of GenAI by organizations like OpenAI and Meta.

    Image 1

    TL;DR 🚅

    @@ -780,12 +795,12 @@

    Blog Posts

  • The word "Open" is used widely within the GenAI field: OpenAI, Open Source LLMs
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Thanks to the Open Source Community for all their efforts! Greetings from PyCon 2024 - +

    Expressing gratitude to the open-source community and sharing experiences from PyConDE & PyData Berlin 2024.

    PyCon 2024

    TL;DR ⏱️

    @@ -796,12 +811,12 @@

    Blog Posts

  • Gratitude to all contributors and supporters
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Who will take care of truly low-resource languages? A good step towards more fair GenAI LLM pricing at OpenAI for Japanese-using people! - +

    Exploring the challenges and recent developments in addressing low-resource languages within the GenAI landscape, with a focus on OpenAI's efforts for the Japanese language.

    Image 1

    TL;DR ⏱️

    @@ -812,12 +827,12 @@

    Blog Posts

  • Concerns about AI ethics and inequality
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • What is your preferred LLM family? And do you start with an already finetuned LLM? Why you have chosen this LLM? I love to hear your perspective! - +

    Understanding the preferences and choices behind selecting specific LLM families and their finetuned variants.

    Which Model

    TL;DR ⏱️

    @@ -829,12 +844,12 @@

    Blog Posts

  • LLM-based GenAI pipelines at Comma Soft AG
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • NVIDIA Benchmark might be WRONG cause it states - You lose money AND LLM inference speed if you add more NVIDIA A100. This NVIDIA Benchmark is NOT reliable. - +

    Analyzing the reliability of NVIDIA's benchmark results and the implications for LLM inference speed and hardware investment.

    Image 1

    TL;DR ⏱️

    @@ -845,12 +860,12 @@

    Blog Posts

  • Questions and further reading on the topic
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Too many LLMs?! How to keep track with all the Open Source Models? Identify the finetuned-masked LLMs and its position within the GenAI landscape! - +

    Navigating the complex landscape of GenAI models can be challenging, but it's crucial to understand the foundational and finetuned models to make informed decisions.

    Image 1

    TL;DR ⏱️

    @@ -861,12 +876,12 @@

    Blog Posts

  • Recommendations for navigating this landscape
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Be careful when you are using LLAMA-2! Legal risks & Sustainability Implications due to LLAMA-2 is (NOT) Open Source. - +

    Important considerations regarding LLAMA-2's legal and sustainability implications.

    Image 1

    TL;DR ⏱️

    @@ -877,12 +892,12 @@

    Blog Posts

  • Meta's restrictions and their broader impact
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • The major players in GenAI are facing challenges with their Generative AIs. GenAI capabilities and security issues related to LLMs Tools • 37C3 Presentation - +

    Challenges and security issues in GenAI and LLMs, highlighted at 37C3.

    Image 1

    TL;DR ⏱️

    @@ -893,12 +908,12 @@

    Blog Posts

  • Categories of threats and challenges in GenAI
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Not prompting in English?... You have Problems!! LLM Language Barriers • Democratizing GenAI and fair pricing - +

    Exploring the challenges of using Generative AI with languages other than English and the implications for cost and performance.

    Image 1

    TL;DR ⏱️

    @@ -909,12 +924,12 @@

    Blog Posts

  • Recommendations for LLM pipelines
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Ever wondered about open source LLLM sizes - 7B, 13B, 70B? - +

    Where do those model sizes come from?... My findings!

    Image 1

    TL;DR ⏱️

    @@ -927,27 +942,12 @@

    Blog Posts

  • Questioning the parameter step sizes of models
    • - (Read More)  (Original-Websource) - - -
  • - DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes - -

    Interesting findings on DALLE's content filtering mechanisms.

    -

    Image 1

    -

    TL;DR ⏱️

    - -
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Evil LLMs available! Break GenAI Alignment through finetuning! - +

    Need for LLM Alignment transparency?

    Image 1

    TL;DR ⏱️

    @@ -958,12 +958,12 @@

    Blog Posts

  • Questions about alignment in LLMs
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • LLAMA2 13B is faster than LLAMA2 7B, according to NVIDIA benchmark! - +

    Interesting findings on NVIDIA's LLAMA 2 benchmark results.

    Image 1

    TL;DR ⏱️

    @@ -974,7 +974,7 @@

    Blog Posts

  • Seeking community insights
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource) @@ -988,7 +988,7 @@

    Blog Posts

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/contact/index.html b/contact/index.html index bb7b170..e7c4a15 100644 --- a/contact/index.html +++ b/contact/index.html @@ -447,7 +447,7 @@

    Contact

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/feed/feed.xml b/feed/feed.xml index 9f371d9..6b64604 100644 --- a/feed/feed.xml +++ b/feed/feed.xml @@ -5,16 +5,16 @@ This is a longer description about your blog. - 2024-09-30T00:00:00Z + 2024-12-27T00:00:00Z https://example.com/ Your Name New DE/EU Open Source LLM 🇪🇺 Teuken 7B, Now Truly Open Source? - - 2024-09-30T00:00:00Z - https://example.com/blog/241110_Teuken_Review/ + + 2024-12-27T00:00:00Z + https://example.com/blog/241127_Teuken_Review/ <p>New Model out of Open-GPT-X developed within Germany from European Perspective</p> <p><picture><source type="image/avif" srcset="https://example.com/img/u966D37VBY-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/u966D37VBY-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/u966D37VBY-800.jpeg" alt="Image 1" width="800" height="749"></picture></p> <p><strong>TL;DR ⏱️</strong></p> @@ -49,9 +49,9 @@ We're Dealing with the Butterfly Effect in LLM Creation Pipelines 🦋🤖 - - 2024-09-30T00:00:00Z - https://example.com/blog/241108_were_dealing_with_the_butterfly_effect_in_llm_creation_pipelines/ + + 2024-11-20T00:00:00Z + https://example.com/blog/241120_were_dealing_with_the_butterfly_effect_in_llm_creation_pipelines/ <p>When we develop novel LLMs, AI-Agents and GenAI Pipelines for Alan.de or within our various other GenAI projects, we're continuously learning about the Butterfly Effect in GenAI creation pipelines and how to mitigate this problem. 👨🏼‍💻 🤯 Every change can have a huge impact on the entire pipeline that creates and executes models</p> <p><picture><source type="image/avif" srcset="https://example.com/img/ODUjemRpOm-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/ODUjemRpOm-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/ODUjemRpOm-800.jpeg" alt="Image 1" width="800" height="800"></picture></p> @@ -84,34 +84,13 @@ </ul> <p>In recent and especially upcoming posts I will address how we Comma Soft AG handle this challenge. If you like to see more of the details please leave me a comment what you think and which questions and ideas you have 💬❤️</p> <p>#generativeai #artificialintelligence #llm #machinelearning #alan</p> - - - - We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! - - 2024-09-30T00:00:00Z - https://example.com/blog/241107_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/ - <p>We will see more and more precise adjustment options in GenAI Tooling</p> -<p><picture><source type="image/avif" srcset="https://example.com/img/DGwfP2s1_T-664.avif 664w"><source type="image/webp" srcset="https://example.com/img/DGwfP2s1_T-664.webp 664w"><img loading="lazy" decoding="async" src="https://example.com/img/DGwfP2s1_T-664.png" alt="Image 1" width="664" height="488"></picture></p> -<p><strong>TL;DR ⏱️</strong></p> -<ul> -<li>GenAI often lacks precise Adjuments</li> -<li>Tools and Models start CLosing the Gap</li> -<li>Major Waste of Time and Resources through inefficient Regenerates</li> -</ul> -<h2 id="control-and-hallucinations-in-generated-videos">Control and hallucinations in generated videos:</h2> -<p>Generative AI that looks nice, I’m wondering if the camera turns multiple laps around the guy in the forest, if the color of the jacket on the back would stay the same without having it explicitly described. What do you think, what would happen? 📹🧥</p> -<h2 id="importance-of-genai-control">Importance of GenAI control:</h2> -<p>I think overall the opportunity to control generations by precise prompts or other interfaces to further specify intermediate results will be insanely important if these technologies should be adapted in a high efficient outcome-oriented business. We cannot waste time for prompts and regenerates! Cause of time and energy 🌱⏰</p> -<p>#artificialintelligence #genai</p> -<p><a href="https://www.linkedin.com/posts/carsten-draschner_ai-runway-3d-activity-7259125095783182336-3bZr?utm_source=share&amp;utm_medium=member_desktop">LinkedIn Post</a></p> New Official OSI "Open Source AI definition" attacks Meta's LLAMA as "Open Washing" - - 2024-09-30T00:00:00Z - https://example.com/blog/241106_new_official_osi_open_source_ai_definition_attacks_metas_llama_as_open_washing/ + + 2024-11-13T00:00:00Z + https://example.com/blog/241113_new_official_osi_open_source_ai_definition_attacks_metas_llama_as_open_washing/ <p>We getting closer what should be considered being open source in GenAI and not only open weight.</p> <p><picture><source type="image/avif" srcset="https://example.com/img/MZY7DxFWA2-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/MZY7DxFWA2-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/MZY7DxFWA2-800.jpeg" alt="Image 1" width="800" height="800"></picture></p> <p><strong>TL;DR 📝</strong></p> @@ -148,7 +127,7 @@ The Uncanny Valley Phenomenon in GenAI Face Synthesis - 2024-09-30T00:00:00Z + 2024-11-05T00:00:00Z https://example.com/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/ <p>GenAI hits Uncanny Valley Problems as we have seen in Animation and Computer Games</p> <p><picture><source type="image/avif" srcset="https://example.com/img/MFdV8_8zRS-946.avif 946w"><source type="image/webp" srcset="https://example.com/img/MFdV8_8zRS-946.webp 946w"><img loading="lazy" decoding="async" src="https://example.com/img/MFdV8_8zRS-946.png" alt="Image 1" width="946" height="870"></picture></p> @@ -185,9 +164,9 @@ A GenAI Solution/Tweak to Rescue ARD tagesschau webpage?! - - 2024-09-30T00:00:00Z - https://example.com/blog/241104_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/ + + 2024-10-25T00:00:00Z + https://example.com/blog/241025_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/ <p>Brainstorming how old regulations meet new GenAI opportunities.</p> <p><picture><source type="image/avif" srcset="https://example.com/img/O_cWA3fM8t-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/O_cWA3fM8t-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/O_cWA3fM8t-800.jpeg" alt="Image 1" width="800" height="800"></picture></p> <p><strong>TL;DR 📝</strong></p> @@ -226,13 +205,34 @@ </ul> <p>Link to 🇩🇪 Tagesschau article: <a href="https://lnkd.in/dkx_RZun">https://lnkd.in/dkx_RZun</a></p> <p>#Rundfunkbeitrag #genai #gez #presse #artificialintelligence</p> + + + + We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! + + 2024-10-16T00:00:00Z + https://example.com/blog/241016_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/ + <p>We will see more and more precise adjustment options in GenAI Tooling</p> +<p><picture><source type="image/avif" srcset="https://example.com/img/DGwfP2s1_T-664.avif 664w"><source type="image/webp" srcset="https://example.com/img/DGwfP2s1_T-664.webp 664w"><img loading="lazy" decoding="async" src="https://example.com/img/DGwfP2s1_T-664.png" alt="Image 1" width="664" height="488"></picture></p> +<p><strong>TL;DR ⏱️</strong></p> +<ul> +<li>GenAI often lacks precise Adjuments</li> +<li>Tools and Models start CLosing the Gap</li> +<li>Major Waste of Time and Resources through inefficient Regenerates</li> +</ul> +<h2 id="control-and-hallucinations-in-generated-videos">Control and hallucinations in generated videos:</h2> +<p>Generative AI that looks nice, I’m wondering if the camera turns multiple laps around the guy in the forest, if the color of the jacket on the back would stay the same without having it explicitly described. What do you think, what would happen? 📹🧥</p> +<h2 id="importance-of-genai-control">Importance of GenAI control:</h2> +<p>I think overall the opportunity to control generations by precise prompts or other interfaces to further specify intermediate results will be insanely important if these technologies should be adapted in a high efficient outcome-oriented business. We cannot waste time for prompts and regenerates! Cause of time and energy 🌱⏰</p> +<p>#artificialintelligence #genai</p> +<p><a href="https://www.linkedin.com/posts/carsten-draschner_ai-runway-3d-activity-7259125095783182336-3bZr?utm_source=share&amp;utm_medium=member_desktop">LinkedIn Post</a></p> Choosing the Right LLM for Your Needs - Key Considerations - - 2024-09-30T00:00:00Z - https://example.com/blog/240910_choosing_the_right_llm_for_your_needs_key_considerations/ + + 2024-10-08T00:00:00Z + https://example.com/blog/241008_choosing_the_right_llm_for_your_needs_key_considerations/ <p>Consider the key factors when selecting a Large Language Model (LLM) for your project.</p> <p><picture><source type="image/avif" srcset="https://example.com/img/TSnOnDgQhc-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/TSnOnDgQhc-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/TSnOnDgQhc-800.jpeg" alt="Image 1" width="800" height="800"></picture></p> <p><strong>TL;DR ⏱️</strong></p> @@ -266,9 +266,9 @@ Stop adding Languages to LLMs! The Potential Drawbacks of Training Multilingual Large Language Models (LLMs) for Performance and Sustainability! - - 2024-09-30T00:00:00Z - https://example.com/blog/240909_stop_adding_languages_to_llms_the_potential/ + + 2024-10-01T00:00:00Z + https://example.com/blog/241001_stop_adding_languages_to_llms_the_potential/ <p>Exploring the downsides of creating multilingual LLMs and their impact on performance and resource utilization.</p> <p><picture><source type="image/avif" srcset="https://example.com/img/JA9fsEHeRy-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/JA9fsEHeRy-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/JA9fsEHeRy-800.jpeg" alt="Image 1" width="800" height="800"></picture></p> <p><strong>TL;DR ⏱️</strong></p> @@ -308,79 +308,102 @@ - Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions? - + Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on. + 2024-09-30T00:00:00Z - https://example.com/blog/240908_where_science_meets_innovation_my_personal/ - <p>Highlights and open questions from the Petersberger Gespräche (PG) 2024, covering AI, energy transition, chip technologies, and more.</p> -<p><picture><source type="image/avif" srcset="https://example.com/img/vuj-G2c0U--2198.avif 2198w"><source type="image/webp" srcset="https://example.com/img/vuj-G2c0U--2198.webp 2198w"><img loading="lazy" decoding="async" src="https://example.com/img/vuj-G2c0U--2198.png" alt="PG 2024 Highlights" width="2198" height="952"></picture></p> + https://example.com/blog/240617_do_we_need_another_genai_solution_how_why/ + <p>An overview of the motivations and technical aspects behind developing our own GenAI solution, Alan, at Comma Soft AG.</p> +<p><picture><source type="image/avif" srcset="https://example.com/img/UXAYbK_4C8-986.avif 986w"><source type="image/webp" srcset="https://example.com/img/UXAYbK_4C8-986.webp 986w"><img loading="lazy" decoding="async" src="https://example.com/img/UXAYbK_4C8-986.png" alt="Alan PDF" width="986" height="668"></picture></p> <p><strong>TL;DR ⏱️</strong></p> <ul> -<li>AI and consciousness discussions</li> -<li>Energy transition and regulatory challenges</li> -<li>Distributed chip technologies in Europe</li> -<li>Generative AI in media</li> -<li>Metaverse applications beyond gaming</li> -</ul> -<h2 id="ai-and-consciousness">🧠 AI and Consciousness 🧠</h2> -<p>Joscha Bach from Liquid AI presented on how advancements in AI could potentially redefine our understanding of consciousness. My open question for him is: How does he intend to reliably measure the achievement of AI consciousness in his efforts?</p> -<h2 id="scientific-chemical-hands-on-how-to-fix-energy-transition">👩🏻‍🔬 Scientific Chemical Hands on how to fix energy transition 👩🏻‍🔬</h2> -<p>As chairman of Alexander von Humboldt Foundation, Robert Schlögl delivered an impressively passionate presentation on energy tech for the climate transition and the phasing out of fossil fuels. Besides scientific derivations, he also highlighted significant regulatory issues that hinder successful implementation. I wonder: How these regulations came about, why they are justified if they are (potentially) not scientifically tenable, and how they can now be resolved?</p> -<h2 id="efficient-distributed-chip-technologies-from-the-heart-of-europe">🇪🇺 Efficient Distributed Chip Technologies from the heart of Europe 🇪🇺</h2> -<p>Christian Mayr from Technische Universität Dresden discussed the potential and developments possible in Dresden. He mentioned the use of clustered chip technologies to operate LLMs. One of my open questions is: How can LLMs be distributed across tens of thousands of chips while still achieving acceptable inferential latencies?</p> -<h2 id="genai-in-media-opportunities-and-risks">📰 GenAI in Media, Opportunities and Risks 📰</h2> -<p>In further discussions with Sibylle Anderl from ZEIT Verlagsgruppe, we explored the potentials and risks of Generative AI in text, image, and video generation, and how reducing anonymity could potentially restore credibility. My open question: Whether there is a reliable way to recognize these outputs, given that word frequency alone might not be a proof?</p> -<h2 id="metaverse-not-only-a-playground">🕹️ Metaverse, not only a Playground? 🕹️</h2> -<p>Nico Michels from Siemens presented the potentials of the Metaverse and digital twins. Q: I'd love to see how such models can help even small municipalities with climate adaptation models, regional impact assessments, and improvement options?</p> -<p>The open questions highlight the stimulating topics the diverse guests who aim to drive positive change with a progressive mindset. I would like to thank Comma Soft AG and everyone involved for organizing this fantastic event, especially Stephan Huthmacher, whose heartfelt dedication makes this recurring, inspiring event possible. ❤️</p> -<p>Share your thoughts and impressions, I'll share the streams 🤗</p> -<p>#artificialintelligence #genai #sustainability #petersbergergespräche</p> -<p><a href="https://www.linkedin.com/posts/carsten-draschner_artificialintelligence-genai-sustainability-activity-7243978164564111361-12R6?utm_source=share&amp;utm_medium=member_desktop">LinkedIn Post</a></p> +<li>Diverse GenAI solutions exist</li> +<li>Unique motivations for developing our own tool</li> +<li>Technical advantages of our solution</li> +<li>Questions on custom development vs. wrappers</li> +</ul> +<h2 id="current-landscape">Current Landscape: 🏞️</h2> +<ul> +<li>There are a variety of GenAI solutions out there</li> +<li>Many models from big tech companies work increasingly well for certain use cases.</li> +<li>And we have launched our own solution. Why?</li> +</ul> +<h2 id="our-technical-perspective">Our technical perspective: 🤓</h2> +<ul> +<li>Data-Flow-Control: We want to know what happens to our data.</li> +<li>Hosting: We host all our models and data on German servers.</li> +<li>On-Premise: We can run our entire tech stack on-premise without any internet connection</li> +<li>Our-own-Alan-Model: We optimize our models for dedicated use cases through further training (e.g., better language skills in GER)</li> +<li>Knowledge-Enrichment: We decide for ourselves whether we want to make new knowledge available through our novel training approach or through our optimized RAG approach (as you might know there are plenty of options).</li> +<li>Alignment: We have control over the alignment, i.e., to what extent the values and behaviors of the model match our expectations.</li> +<li>SOTA: We always remain state of the art in a fusion of existing open source base models and in-house developments.</li> +<li>Benchmarking: We use benchmarks and measures we trust.</li> +</ul> +<p>With a great team at Comma Soft AG, it's enjoyable to develop such a helpful tool ❤️ In the appended PDF we show an extract of the full text story of our ALAN development journey 📖</p> +<h2 id="questions">Questions: 🤔</h2> +<ul> +<li>How did you decide if you want to develop something yourself or use more kind of wrappers?</li> +<li>How important is the management of sensitive data for you and where are your servers located?</li> +</ul> +<h2 id="literature">Literature: 📚</h2> +<ul> +<li>Here is the full Humboldt Travel Report with more background stories: <a href="https://lnkd.in/emwvpQsW">https://lnkd.in/emwvpQsW</a></li> +<li>Adjusting Model Alignment <a href="https://lnkd.in/eWS-VZCD">https://lnkd.in/eWS-VZCD</a></li> +<li>Model Selection <a href="https://lnkd.in/duGzWugD">https://lnkd.in/duGzWugD</a></li> +<li>Model Benchmarking <a href="https://lnkd.in/dmBeQZ_j">https://lnkd.in/dmBeQZ_j</a></li> +</ul> +<p>For more GenAI LLM content and R&amp;D discussions, follow me on LinkedIn. If you want to try out Alan, reach out to me: <a href="http://alan.de">http://alan.de</a></p> +<p>#artificialintelligence #llm #machinelearning #handson #genai</p> +<p><a href="https://www.linkedin.com/posts/carsten-draschner_humboldt-travel-report-alan-llm-genai-activity-7208854388101005312-Xr_o?utm_source=share&amp;utm_medium=member_desktop">LinkedIn Post</a></p> - Today's Research Proposal - How to achieve "real" thinking and reasoning in GenAI, rather than just relying on a silent Chain of Thought, as seen in ReflectionAI or possibly GPT-o1? - + DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes + 2024-09-30T00:00:00Z - https://example.com/blog/240907_todays_research_proposal_how_to_achieve_real/ - <p>Exploring the potential for achieving true reasoning and thinking in Generative AI models beyond the current Chain of Thought methodologies.</p> -<p><picture><source type="image/avif" srcset="https://example.com/img/W1uRDiZMha-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/W1uRDiZMha-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/W1uRDiZMha-800.jpeg" alt="Image 1" width="800" height="696"></picture></p> + https://example.com/blog/240213_dalle_has_surprising_guardrails/ + <p>Interesting findings on DALLE's content filtering mechanisms.</p> +<p><picture><source type="image/avif" srcset="https://example.com/img/o3BtZuF2Rz-800.avif 800w"><source type="image/webp" srcset="https://example.com/img/o3BtZuF2Rz-800.webp 800w"><img loading="lazy" decoding="async" src="https://example.com/img/o3BtZuF2Rz-800.jpeg" alt="Image 1" width="800" height="800"></picture></p> <p><strong>TL;DR ⏱️</strong></p> <ul> -<li>Current state of reasoning in models</li> -<li>Possibilities for transformers to learn to think</li> -<li>Customization ideas for achieving true reasoning</li> -<li>Open questions and discussion points</li> -</ul> -<h2 id="currently-reasoning-more-through-silent-the-cot">Currently, reasoning more through silent the CoT 😥</h2> -<ul> -<li>There are currently models that claim to possess reasoning capabilities.</li> -<li>However, this is more about the Chain of Thought and the use of special tokens.</li> -<li>These enable the model to generate more text, thereby increasing the stability of its answers.</li> -<li>This process of &quot;reasoning&quot; is not transparently/barely shown.</li> -<li>I believe this is less about reasoning or thinking and more about statistical stability.</li> -<li>Moreover, it is not possible to observe the reasoning process, and it is often difficult to identify when a model has gone astray.</li> -</ul> -<h2 id="how-could-transformers-learn-to-think">How could transformers learn to think? 🤯</h2> -<ul> -<li>Why can I still imagine that it is possible to learn to think or reason with the transformer architecture?</li> -<li>Due to the transformer architecture, which includes tokenization, attention, and multi-layer perceptron elements, as well as the universal function approximation hypothesis, the following can be imagined: -<ol> -<li>In early layers, the network learns something akin to named entity recognition.</li> -<li>Later layers and embedding dimensions create structures that learn propositional logic and structures similar to knowledge graphs or their embeddings.</li> -<li>Finally, based on this, resonating and reflecting completeness of output logic components could be achieved.</li> -</ol> -</li> -</ul> -<h2 id="what-else-would-i-customize-imho">What else would I customize? 👨🏼‍💻 (IMHO)</h2> -<p>I would find it interesting to combine this with thought-diffusion models, making hierarchical planning possible. I have several more ideas..., reach out to me if you like to start a discussion ❤️</p> -<h2 id="questions">Questions:</h2> -<p>Are you aware of any approaches or research projects that attempt this? -What are your thoughts on that, how would you build &quot;AGI&quot; it?</p> -<p>I had great idea exchanges on this with colleagues from Comma Soft AG, Lamarr-Institut &amp; Smart Data Analytics.</p> -<p>Let's not waste money and time on believing in marketing claims and rebranding of the word &quot;reasoning&quot;, but let's start to think of how it could actually be achieved 🤗</p> -<p>#AGI #GenAI #artificialintelligence #research</p> -<p><a href="https://www.linkedin.com/posts/carsten-draschner_artificialintelligence-genai-sustainability-activity-7243978164564111361-12R6?utm_source=share&amp;utm_medium=member_desktop">LinkedIn Post</a></p> +<li>DALLE-3 filters your content AFTER image creation</li> +<li>With prompt “dead cookies” you can reproduce inconsistent filtering over OpenAI API</li> +<li>40% of cases with same “dead cookies” prompt stop through content filter and 60% reach us over API</li> +</ul> +<h2 id="what-is-dalle-3">What is DALLE-3 🖼️</h2> +<ul> +<li>DALLE 3 is a generative text-to-image model by OpenAI also available as API</li> +<li>You pay per image</li> +<li>Images are created based on your prompt like “dead cookies”.</li> +<li>You can also add details like: “Dead Cookies in cute Pixar style” or “Dead cookies with dramatic situation in cute Pixar style”</li> +<li>Open-Source image GenAI models alternatives are available e.g., Stable Diffusion</li> +<li>Image GenAI are under discussion because of misuse like deepfakes or reproducing intellectual property.</li> +</ul> +<h2 id="finding-observation">Finding/Observation: 👩🏽‍🔬</h2> +<ul> +<li>DALLE-3 has a content filter to reduce misuse</li> +<li>If you hit the content filter you do not get a resulting image for your prompt.</li> +<li>The content filter is not applied based on the prompt, it is applied AFTER DALLE-3 generated the image, and the API decides in an extra step if the image should be sent to you. Likely some Image classifier.</li> +<li>Same prompt sometimes results in an image and sometimes in a content-filter response. For the prompt “dead cookies” you get in 60% of requests an image and in 40% a content filter issue</li> +</ul> +<h2 id="how-we-found-out">How we found out 🍪</h2> +<ul> +<li>We @Comma Soft AG develop tools and pipelines with OS GenAI but also with API requests.</li> +<li>For good API-response handling, we also had to consider content filter scenarios. So we combined trigger words like &quot;dead&quot; with something like &quot;cookies&quot;</li> +<li>We had inconsistent content filter and still the finding that in case of content filter the response time was roughly as long as in the case of created image.</li> +</ul> +<h2 id="my-questions-to-you">My Questions to you 🤷🏼‍♂️</h2> +<ul> +<li>Who should pay for “dead cookies” if the resulting image was created but not sent due to content filter?</li> +<li>Have you known that the content filter for DALLE-3 is applied after image generation?</li> +<li>Do you also encounter content filter although your prompts were in principle ok?</li> +<li>Do you think content filters are a reasonable image GenAI misuse countermeasure?</li> +<li>How would you reduce Image GenAI misuse?</li> +<li>And most interesting (and we might never know OpenAI/DALL-E Open Ai), how do “Dead Cookies” images look like which are filtered out? 😅</li> +</ul> +<p>The image was created by the prompt &quot;Dead cookies in cute pixar style&quot; +If you like more of such content, reach out to me 😊</p> +<p>#artificalintelligence #genai #aiethics #dalle #openai #texttoimage #deepfakes</p> +<p><a href="https://www.linkedin.com/posts/carsten-draschner_artificalintelligence-genai-aiethics-activity-7158865169689821184-J8Y_?utm_source=share&amp;utm_medium=member_desktop">LinkedIn Post</a></p> \ No newline at end of file diff --git a/index.html b/index.html index 29a0a8b..b2a5f2c 100644 --- a/index.html +++ b/index.html @@ -469,7 +469,7 @@

    Home

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/projects/2016_Chip_VR/index.html b/projects/2016_Chip_VR/index.html index 3ca0520..882efa2 100644 --- a/projects/2016_Chip_VR/index.html +++ b/projects/2016_Chip_VR/index.html @@ -600,7 +600,7 @@

    Modern Chip Design

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/projects/2019_Menu_CV/index.html b/projects/2019_Menu_CV/index.html index a8748ea..4ac3da6 100644 --- a/projects/2019_Menu_CV/index.html +++ b/projects/2019_Menu_CV/index.html @@ -591,7 +591,7 @@

    How to find the

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/projects/2022_SANSA/index.html b/projects/2022_SANSA/index.html index a2cbdf0..d1ba1f7 100644 --- a/projects/2022_SANSA/index.html +++ b/projects/2022_SANSA/index.html @@ -591,7 +591,7 @@

    diff --git a/projects/index.html b/projects/index.html index b0196cd..6fea46f 100644 --- a/projects/index.html +++ b/projects/index.html @@ -442,7 +442,7 @@

    Projects

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/research/160901_bachelor_thesis_the_development_of_complexity_in_chip_design_and_its_visualization_within_virtual_reality/index.html b/research/160901_bachelor_thesis_the_development_of_complexity_in_chip_design_and_its_visualization_within_virtual_reality/index.html index 9af11c7..f8f0f4e 100644 --- a/research/160901_bachelor_thesis_the_development_of_complexity_in_chip_design_and_its_visualization_within_virtual_reality/index.html +++ b/research/160901_bachelor_thesis_the_development_of_complexity_in_chip_design_and_its_visualization_within_virtual_reality/index.html @@ -595,7 +595,7 @@

    diff --git a/research/190401_smart_chef_evolving_recipes/index.html b/research/190401_smart_chef_evolving_recipes/index.html index 694ef37..880decb 100644 --- a/research/190401_smart_chef_evolving_recipes/index.html +++ b/research/190401_smart_chef_evolving_recipes/index.html @@ -595,7 +595,7 @@

    Smart Chef - Evolving Recipes

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/research/200901_distsim_scalable_distributed_in_memory/index.html b/research/200901_distsim_scalable_distributed_in_memory/index.html index 62a87a2..28e5eb3 100644 --- a/research/200901_distsim_scalable_distributed_in_memory/index.html +++ b/research/200901_distsim_scalable_distributed_in_memory/index.html @@ -594,7 +594,7 @@

    diff --git a/research/210401_literal2feature/index.html b/research/210401_literal2feature/index.html index 80aff2c..23a3cc8 100644 --- a/research/210401_literal2feature/index.html +++ b/research/210401_literal2feature/index.html @@ -594,7 +594,7 @@

    Liter

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/research/210501_distrdf2ml/index.html b/research/210501_distrdf2ml/index.html index ac8f8a9..df848c5 100644 --- a/research/210501_distrdf2ml/index.html +++ b/research/210501_distrdf2ml/index.html @@ -594,7 +594,7 @@

    diff --git a/research/220902_ethical_and_sustainability/index.html b/research/220902_ethical_and_sustainability/index.html index ab09b86..abf8dcf 100644 --- a/research/220902_ethical_and_sustainability/index.html +++ b/research/220902_ethical_and_sustainability/index.html @@ -594,7 +594,7 @@

    diff --git a/research/221214_scalable_distributed_machine_learning_for_knowledge_graphs/index.html b/research/221214_scalable_distributed_machine_learning_for_knowledge_graphs/index.html index 402b0ff..c417dc0 100644 --- a/research/221214_scalable_distributed_machine_learning_for_knowledge_graphs/index.html +++ b/research/221214_scalable_distributed_machine_learning_for_knowledge_graphs/index.html @@ -584,7 +584,7 @@

    Scalable Dis

    Within this work, we developed novel approaches for Machine Learning on Knowledge Graphs while considering ethical and sustainability dimensions. In particular, we developed technologies that create fixed-length numeric feature vectors. These include methods that, like graph kernels, extract features from the graph in the context of the map-reduce operations relevant for distributed computation. The feature extraction also includes the multi-modal data of KG literals. Accordingly, we have developed methods that enable SPARQL-based feature extraction and assist in creating complex feature-extracting queries. Based on these extracted features, we further contributed scalable, distributed, and explainable ML and data analytics methods such as semantic similarity estimation and classification or regression ML pipelines demonstrating noticeable performance. We support the transparency, reusability, and reproducibility of our novel open-source approaches by results and meta-data semantification. This semantification transfers the original graph data with the hyper-parameter setup and explainability information, in addition to the predicted results of the ML pipelines, into a semantic native KG. Due to the technological complexity, we enable the application of our algorithm technologies through complementary work such as the use in coding notebooks and the use in Rest API-based environments. Our work also describes the multidimensional and interwoven optimization dimensions of ethical and sustainable KG-based ML. We extended the existing technology stack SANSA, which is used for distributed processing and native semantic data handling, by several scientific publications and software framework releases to offer these functionalities for distributed ML on KGs.

    -
    • ← Previous
      SimE4KG - Distributed Explainable multi-modal Semantic Similarity Estimation for Knowledge Graphs
    • Next →
      LLAMA2 13B is faster than LLAMA2 7B, according to NVIDIA benchmark!
      • + @@ -594,7 +594,7 @@

      Scalable Dis

      © 2024 Carsten Felix Draschner, PhD

      - + diff --git a/research/230701_sime4kg/index.html b/research/230701_sime4kg/index.html index d7ea80e..0f39ad5 100644 --- a/research/230701_sime4kg/index.html +++ b/research/230701_sime4kg/index.html @@ -594,7 +594,7 @@

      diff --git a/research/index.html b/research/index.html index a52ea39..7ca3d97 100644 --- a/research/index.html +++ b/research/index.html @@ -537,7 +537,7 @@

      Research

      © 2024 Carsten Felix Draschner, PhD

      - + diff --git a/sitemap.xml b/sitemap.xml index c5055bd..2a2dcc7 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -57,183 +57,173 @@ - https://example.com/blog/240702_llama2_13b_is_faster_than_llama2_7b/ - 2024-09-30 + https://example.com/blog/240202_llama2_13b_is_faster_than_llama2_7b/ + 2024-02-05 - https://example.com/blog/240703_evil_llms_available_break_genai/ - 2024-09-30 + https://example.com/blog/240205_evil_llms_available_break_genai/ + 2024-02-05 - https://example.com/blog/240704_dalle_has_surprising_guardrails/ - 2024-09-30 + https://example.com/blog/240227_ever_wondered_about_open_source_llm_sizes/ + 2024-02-27 - https://example.com/blog/240705_ever_wondered_about_open_source_llm_sizes/ - 2024-09-30 + https://example.com/blog/240307_not_prompting_in_english/ + 2024-03-07 - https://example.com/blog/240707_not_prompting_in_english/ - 2024-09-30 + https://example.com/blog/240313_the_major_players_in_genai_are_facing_challenges/ + 2024-03-13 - https://example.com/blog/240708_the_major_players_in_genai_are_facing_challenges/ - 2024-09-30 + https://example.com/blog/240327_be_careful_when_you_are_using_llama/ + 2024-03-27 - https://example.com/blog/240709_be_careful_when_you_are_using_llama/ - 2024-09-30 + https://example.com/blog/240408_too_many_llms_how_to_keep_track_with_all/ + 2024-04-08 - https://example.com/blog/240709_too_many_llms_how_to_keep_track_with_all/ - 2024-09-30 + https://example.com/blog/240415_nvidia_benchmark_might_be_wrong/ + 2024-04-15 - https://example.com/blog/240710_nvidia_benchmark_might_be_wrong/ - 2024-09-30 + https://example.com/blog/240422_what_is_your_preferred_llm_family/ + 2024-04-22 - https://example.com/blog/240711_what_is_your_preferred_llm_family/ - 2024-09-30 + https://example.com/blog/240422_who_will_take_care_of_truly_low_resource_languages/ + 2024-04-29 - https://example.com/blog/240712_who_will_take_care_of_truly_low_resource_languages/ - 2024-09-30 + https://example.com/blog/240506_thanks_to_the_open_source_community/ + 2024-05-06 - https://example.com/blog/240801_thanks_to_the_open_source_community/ - 2024-09-30 + https://example.com/blog/240513_be_careful_when_you_speak_of_open_source_genai/ + 2024-05-13 - https://example.com/blog/240802_be_careful_when_you_speak_of_open_source_genai/ - 2024-09-30 + https://example.com/blog/240520_what_expectations_do_you_have_regarding_the_values_and_norms/ + 2024-05-20 - https://example.com/blog/240803_what_expectations_do_you_have_regarding_the_values_and_norms/ - 2024-09-30 + https://example.com/blog/240603_do_you_differentiate_ai_ethics/ + 2024-06-03 - https://example.com/blog/240804_do_you_differentiate_ai_ethics/ - 2024-09-30 + https://example.com/blog/240610_will_we_reach_agi_and_if/ + 2024-06-10 - https://example.com/blog/240805_will_we_reach_agi_and_if/ - 2024-09-30 + https://example.com/blog/240624_alternative_to_genai_creativity_watch_and_try/ + 2024-06-24 - https://example.com/blog/240806_do_we_need_another_genai_solution_how_why/ - 2024-09-30 + https://example.com/blog/240701_adjust_genai_responses_towards_more_ethical_behavior_possible/ + 2024-07-01 - https://example.com/blog/240807_alternative_to_genai_creativity_watch_and_try/ - 2024-09-30 + https://example.com/blog/240708_genai_what_is_plagiarism_its_impact/ + 2024-07-08 - https://example.com/blog/240808_adjust_genai_responses_towards_more_ethical_behavior_possible/ - 2024-09-30 + https://example.com/blog/240715_llms_big_vs_small_bigger_is_better_or_lets_not_waste_energy/ + 2024-07-15 - https://example.com/blog/240809_genai_what_is_plagiarism_its_impact/ - 2024-09-30 + https://example.com/blog/240722_these_results_give_me_hope_for_sustainable_ai/ + 2024-07-22 - https://example.com/blog/240901_llms_big_vs_small_bigger_is_better_or_lets_not_waste_energy/ - 2024-09-30 + https://example.com/blog/240729_what_is_the_perfect_approach_to_adjust_an_llm_to_your_genai_use_case/ + 2024-07-29 - https://example.com/blog/240902_these_results_give_me_hope_for_sustainable_ai/ - 2024-09-30 + https://example.com/blog/240830_combining_the_hugging_face_model_platform_and_knowledge/ + 2024-08-30 - https://example.com/blog/240903_what_is_the_perfect_approach_to_adjust_an_llm_to_your_genai_use_case/ - 2024-09-30 + https://example.com/blog/240903_sustainable_air_gapped_on_prem_llm_solution/ + 2024-09-03 - https://example.com/blog/240904_combining_the_hugging_face_model_platform_and_knowledge/ - 2024-09-30 + https://example.com/blog/240910_for_more_sustainability_transparency_in_genai_share/ + 2024-09-10 - https://example.com/blog/240905_%20sustainable_air_gapped_on_prem_llm_solution/ - 2024-09-30 + https://example.com/blog/240917_todays_research_proposal_how_to_achieve_real/ + 2024-09-17 - https://example.com/blog/240906_for_more_sustainability_transparency_in_genai_share/ - 2024-09-30 - - - - https://example.com/blog/240907_todays_research_proposal_how_to_achieve_real/ - 2024-09-30 - - - - https://example.com/blog/240908_where_science_meets_innovation_my_personal/ - 2024-09-30 + https://example.com/blog/240924_where_science_meets_innovation_my_personal/ + 2024-09-24 - https://example.com/blog/240909_stop_adding_languages_to_llms_the_potential/ + https://example.com/blog/240213_dalle_has_surprising_guardrails/ 2024-09-30 - https://example.com/blog/240910_choosing_the_right_llm_for_your_needs_key_considerations/ + https://example.com/blog/240617_do_we_need_another_genai_solution_how_why/ 2024-09-30 - https://example.com/blog/241104_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/ - 2024-09-30 + https://example.com/blog/241001_stop_adding_languages_to_llms_the_potential/ + 2024-10-01 - https://example.com/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/ - 2024-09-30 + https://example.com/blog/241008_choosing_the_right_llm_for_your_needs_key_considerations/ + 2024-10-08 - https://example.com/blog/241106_new_official_osi_open_source_ai_definition_attacks_metas_llama_as_open_washing/ - 2024-09-30 + https://example.com/blog/241016_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/ + 2024-10-16 - https://example.com/blog/241107_we_cannot_waste_time_and_resources_through_regenerating_genai_creations_genai_control_is_key_for_productivity/ - 2024-09-30 + https://example.com/blog/241025_a_genai_solution_tweak_to_rescue_ard_tagesschau_webpage/ + 2024-10-25 - https://example.com/blog/241108_were_dealing_with_the_butterfly_effect_in_llm_creation_pipelines/ - 2024-09-30 + https://example.com/blog/241105_the_uncanny_valley_phenomenon_in_genai_face_synthesis/ + 2024-11-05 - https://example.com/blog/241110_Teuken_Review/ - 2024-09-30 + https://example.com/blog/241113_new_official_osi_open_source_ai_definition_attacks_metas_llama_as_open_washing/ + 2024-11-13 @@ -266,6 +256,11 @@ 2024-11-13 + + https://example.com/blog/241120_were_dealing_with_the_butterfly_effect_in_llm_creation_pipelines/ + 2024-11-20 + + https://example.com/contact/ 2024-11-27 @@ -273,6 +268,11 @@ https://example.com/feed/feed.xml - 2024-12-02 + 2024-12-03 + + + + https://example.com/blog/241127_Teuken_Review/ + 2024-12-27 diff --git a/tags/blog/index.html b/tags/blog/index.html index 9c59b18..91f4081 100644 --- a/tags/blog/index.html +++ b/tags/blog/index.html @@ -406,7 +406,7 @@

      Tagged “blog”

    • New DE/EU Open Source LLM 🇪🇺 Teuken 7B, Now Truly Open Source? - +

      New Model out of Open-GPT-X developed within Germany from European Perspective

      Image 1

      TL;DR ⏱️

      @@ -416,12 +416,12 @@

      Tagged “blog”

    • Problems in Alignment?
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • We're Dealing with the Butterfly Effect in LLM Creation Pipelines 🦋🤖 - +

    When we develop novel LLMs, AI-Agents and GenAI Pipelines for Alan.de or within our various other GenAI projects, we're continuously learning about the Butterfly Effect in GenAI creation pipelines and how to mitigate this problem. 👨🏼‍💻 🤯 Every change can have a huge impact on the entire pipeline that creates and executes models

    Image 1

    @@ -433,27 +433,12 @@

    Tagged “blog”

  • Strategies to mitigate the Butterfly Effect are essential for optimal performance.
    • - (Read More)  (Original-Websource) - - -
  • - We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! - -

    We will see more and more precise adjustment options in GenAI Tooling

    -

    Image 1

    -

    TL;DR ⏱️

    -
      -
    • GenAI often lacks precise Adjuments
      • -
    • Tools and Models start CLosing the Gap
      • -
    • Major Waste of Time and Resources through inefficient Regenerates
      • -
    -
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • New Official OSI "Open Source AI definition" attacks Meta's LLAMA as "Open Washing" - +

    We getting closer what should be considered being open source in GenAI and not only open weight.

    Image 1

    TL;DR 📝

    @@ -464,12 +449,12 @@

    Tagged “blog”

  • "Open Washing" Problem of Meta and others?!
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • The Uncanny Valley Phenomenon in GenAI Face Synthesis - +

    GenAI hits Uncanny Valley Problems as we have seen in Animation and Computer Games

    Image 1

    TL;DR 📝

    @@ -485,7 +470,7 @@

    Tagged “blog”

  • A GenAI Solution/Tweak to Rescue ARD tagesschau webpage?! - +

    Brainstorming how old regulations meet new GenAI opportunities.

    Image 1

    TL;DR 📝

    @@ -496,12 +481,27 @@

    Tagged “blog”

  • Questions about the impact and regulation of GenAI in public broadcasting contexts.
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource) + + +
  • + We Cannot Waste Time and Resources through Regenerating GenAI Creations! GenAI Control is Key for Productivity! + +

    We will see more and more precise adjustment options in GenAI Tooling

    +

    Image 1

    +

    TL;DR ⏱️

    +
      +
    • GenAI often lacks precise Adjuments
      • +
    • Tools and Models start CLosing the Gap
      • +
    • Major Waste of Time and Resources through inefficient Regenerates
      • +
    +
    + (Read More)  (Original-Websource)
  • Choosing the Right LLM for Your Needs - Key Considerations - +

    Consider the key factors when selecting a Large Language Model (LLM) for your project.

    Image 1

    TL;DR ⏱️

    @@ -512,12 +512,12 @@

    Tagged “blog”

  • Alignment
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Stop adding Languages to LLMs! The Potential Drawbacks of Training Multilingual Large Language Models (LLMs) for Performance and Sustainability! - +

    Exploring the downsides of creating multilingual LLMs and their impact on performance and resource utilization.

    Image 1

    TL;DR ⏱️

    @@ -528,12 +528,43 @@

    Tagged “blog”

  • Weighing multilingual models against language-specific models
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • - Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions? + Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on. +

    An overview of the motivations and technical aspects behind developing our own GenAI solution, Alan, at Comma Soft AG.

    +

    Alan PDF

    +

    TL;DR ⏱️

    +
      +
    • Diverse GenAI solutions exist
      • +
    • Unique motivations for developing our own tool
      • +
    • Technical advantages of our solution
      • +
    • Questions on custom development vs. wrappers
      • +
    +
    + (Read More)  (Original-Websource) +
    • + +
  • + DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes + +

    Interesting findings on DALLE's content filtering mechanisms.

    +

    Image 1

    +

    TL;DR ⏱️

    +
      +
    • DALLE-3 filters your content AFTER image creation
      • +
    • With prompt “dead cookies” you can reproduce inconsistent filtering over OpenAI API
      • +
    • 40% of cases with same “dead cookies” prompt stop through content filter and 60% reach us over API
      • +
    +
    + (Read More)  (Original-Websource) +
    • + +
  • + Where Science Meets Innovation - My personal Highlights & Insights into the PG 2024! Do you have answers to the open Questions? +

    Highlights and open questions from the Petersberger Gespräche (PG) 2024, covering AI, energy transition, chip technologies, and more.

    PG 2024 Highlights

    TL;DR ⏱️

    @@ -545,12 +576,12 @@

    Tagged “blog”

  • Metaverse applications beyond gaming
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Today's Research Proposal - How to achieve "real" thinking and reasoning in GenAI, rather than just relying on a silent Chain of Thought, as seen in ReflectionAI or possibly GPT-o1? - +

    Exploring the potential for achieving true reasoning and thinking in Generative AI models beyond the current Chain of Thought methodologies.

    Image 1

    TL;DR ⏱️

    @@ -561,12 +592,12 @@

    Tagged “blog”

  • Open questions and discussion points
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • For more sustainability transparency in GenAI! Share your knowledge and reduce energy waste! - +

    Emphasizing the importance of transparency and shared knowledge to enhance sustainability in GenAI.

    Image 1

    TL;DR ⏱️

    @@ -576,12 +607,12 @@

    Tagged “blog”

  • Open source models can optimize future development
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Sustainable Air-Gapped On-Prem LLM Solution! How can we make GenAI available on almost any hardware, and how is it also available as a portable demo on our Alan Notebook - +

    Exploring the development of a full-stack GenAI LLM solution that can run on a variety of hardware configurations, including a portable demo setup.

    Alan Notebook

    TL;DR ⏱️

    @@ -592,12 +623,12 @@

    Tagged “blog”

  • Portable demo available on Alan Notebook
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Combining the Hugging Face Model Platform and Knowledge Graph trend analysis over time could improve GenAI research and reduce waste of energy! - +

    Exploring the potential of leveraging knowledge graphs to analyze trends in evolving models for better GenAI research and efficiency.

    Image 1

    TL;DR ⏱️

    @@ -607,12 +638,12 @@

    Tagged “blog”

  • Potential for a crowd-sourced GenAI cookbook
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • What is the perfect approach to adjust an LLM to your GenAI use case? - +

    Exploring various methods to customize LLMs for specific GenAI use cases, ranging from simple to complex approaches.

    Model Training

    TL;DR ⏱️

    @@ -623,12 +654,12 @@

    Tagged “blog”

  • More dimensions to improve GenAI use cases
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • These results give me hope for sustainable AI 🌱 - +

    I'm impressed by some of the recent advances in the field of "small" open-weight Language Models (LLMs).

    Sustainable AI

    TL;DR ⏱️

    @@ -638,12 +669,12 @@

    Tagged “blog”

  • Model distillation reduces hardware needs
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • LLMs - Big vs Small. Bigger is Better!? OR Let's not waste energy!? - +

    The AI community is abuzz with debates over the efficacy of large versus small language models. Both have their own merits and limitations.

    Model Size

    TL;DR ⏱️

    @@ -654,12 +685,12 @@

    Tagged “blog”

  • Links to further reading
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • GenAI, what is plagiarism? Its Impact on Science. How should it be handled? What is your perspective? - +

    Discussing the implications of GenAI on scientific work and the thin line between acceptable use and plagiarism.

    Image 1

    TL;DR ⏱️

    @@ -671,12 +702,12 @@

    Tagged “blog”

  • Opportunities for better research
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Adjust GenAI responses towards more ethical behavior possible through system prompts!? Do you trust in such LLM-chat prepended pamphlets? - +

    Exploring the potential and challenges of using system prompts to guide LLM behavior towards ethical outputs.

    Ethical AI

    TL;DR ⏱️

    @@ -687,12 +718,12 @@

    Tagged “blog”

  • Questions on designing and revealing system prompts
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Alternative to GenAI creativity? Watch and try out these fun Evolutionary Algorithms. Problem-solving without GenAI and SGD-based approaches explained! - +

    Exploring Evolutionary Algorithms as an alternative to GenAI for problem-solving, using a fun 2D vehicle race example.

    Evo Cars

    TL;DR ⏱️

    @@ -703,28 +734,12 @@

    Tagged “blog”

  • Steps involved in EAs explained
    • - (Read More)  (Original-Websource) - - -
  • - Do we need another GenAI solution? How & why we developed a full-stack GenAI LLM+RAG tool called Alan. A sneak peek at what I am currently working on. - -

    An overview of the motivations and technical aspects behind developing our own GenAI solution, Alan, at Comma Soft AG.

    -

    Alan PDF

    -

    TL;DR ⏱️

    -
      -
    • Diverse GenAI solutions exist
      • -
    • Unique motivations for developing our own tool
      • -
    • Technical advantages of our solution
      • -
    • Questions on custom development vs. wrappers
      • -
    -
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Will we reach AGI, and if so, by transformers-based architectures? Share your perspective! - +

    Exploring the potential of transformers-based architectures in achieving Artificial General Intelligence (AGI) and the ongoing debate surrounding it.

    AGI Target

    TL;DR ⏱️

    @@ -734,12 +749,12 @@

    Tagged “blog”

  • Optimistic yet cautious approach at Comma Soft AG
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Do you differentiate AI Ethics principles between AI/ML fields like GenAI/LLMs or Knowledge Graph-based ML? How do we deal with so-called facts on the internet as training data? - +

    Exploring the nuances of AI ethics across different AI/ML fields and handling internet-based training data responsibly.

    Ethics in AI

    TL;DR ⏱️

    @@ -750,12 +765,12 @@

    Tagged “blog”

  • Questions to ponder on AI ethics practices
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • What expectations do you have regarding the values and norms of your GenAI chat assistants? Highly sensitive topic in the LLM space! My take... - +

    Exploring the ethical considerations and expectations surrounding the values and norms embedded in GenAI chat assistants.

    Image 1

    TL;DR ⏱️

    @@ -766,12 +781,12 @@

    Tagged “blog”

  • Need for a holistic view on model behavior
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Be careful when you speak of Open (Source) GenAI. Why OpenAI and Meta (shouldn't) use the word Open within their GenAI efforts? - +

    Examining the implications of using the term "Open" in the context of GenAI by organizations like OpenAI and Meta.

    Image 1

    TL;DR 🚅

    @@ -780,12 +795,12 @@

    Tagged “blog”

  • The word "Open" is used widely within the GenAI field: OpenAI, Open Source LLMs
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Thanks to the Open Source Community for all their efforts! Greetings from PyCon 2024 - +

    Expressing gratitude to the open-source community and sharing experiences from PyConDE & PyData Berlin 2024.

    PyCon 2024

    TL;DR ⏱️

    @@ -796,12 +811,12 @@

    Tagged “blog”

  • Gratitude to all contributors and supporters
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Who will take care of truly low-resource languages? A good step towards more fair GenAI LLM pricing at OpenAI for Japanese-using people! - +

    Exploring the challenges and recent developments in addressing low-resource languages within the GenAI landscape, with a focus on OpenAI's efforts for the Japanese language.

    Image 1

    TL;DR ⏱️

    @@ -812,12 +827,12 @@

    Tagged “blog”

  • Concerns about AI ethics and inequality
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • What is your preferred LLM family? And do you start with an already finetuned LLM? Why you have chosen this LLM? I love to hear your perspective! - +

    Understanding the preferences and choices behind selecting specific LLM families and their finetuned variants.

    Which Model

    TL;DR ⏱️

    @@ -829,12 +844,12 @@

    Tagged “blog”

  • LLM-based GenAI pipelines at Comma Soft AG
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • NVIDIA Benchmark might be WRONG cause it states - You lose money AND LLM inference speed if you add more NVIDIA A100. This NVIDIA Benchmark is NOT reliable. - +

    Analyzing the reliability of NVIDIA's benchmark results and the implications for LLM inference speed and hardware investment.

    Image 1

    TL;DR ⏱️

    @@ -845,12 +860,12 @@

    Tagged “blog”

  • Questions and further reading on the topic
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Too many LLMs?! How to keep track with all the Open Source Models? Identify the finetuned-masked LLMs and its position within the GenAI landscape! - +

    Navigating the complex landscape of GenAI models can be challenging, but it's crucial to understand the foundational and finetuned models to make informed decisions.

    Image 1

    TL;DR ⏱️

    @@ -861,12 +876,12 @@

    Tagged “blog”

  • Recommendations for navigating this landscape
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Be careful when you are using LLAMA-2! Legal risks & Sustainability Implications due to LLAMA-2 is (NOT) Open Source. - +

    Important considerations regarding LLAMA-2's legal and sustainability implications.

    Image 1

    TL;DR ⏱️

    @@ -877,12 +892,12 @@

    Tagged “blog”

  • Meta's restrictions and their broader impact
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • The major players in GenAI are facing challenges with their Generative AIs. GenAI capabilities and security issues related to LLMs Tools • 37C3 Presentation - +

    Challenges and security issues in GenAI and LLMs, highlighted at 37C3.

    Image 1

    TL;DR ⏱️

    @@ -893,12 +908,12 @@

    Tagged “blog”

  • Categories of threats and challenges in GenAI
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Not prompting in English?... You have Problems!! LLM Language Barriers • Democratizing GenAI and fair pricing - +

    Exploring the challenges of using Generative AI with languages other than English and the implications for cost and performance.

    Image 1

    TL;DR ⏱️

    @@ -909,12 +924,12 @@

    Tagged “blog”

  • Recommendations for LLM pipelines
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Ever wondered about open source LLLM sizes - 7B, 13B, 70B? - +

    Where do those model sizes come from?... My findings!

    Image 1

    TL;DR ⏱️

    @@ -927,27 +942,12 @@

    Tagged “blog”

  • Questioning the parameter step sizes of models
    • - (Read More)  (Original-Websource) - - -
  • - DALLE has surprising guardrails. Your image is not filtered based on your prompt. "Dead cookies" may be generated ...sometimes - -

    Interesting findings on DALLE's content filtering mechanisms.

    -

    Image 1

    -

    TL;DR ⏱️

    -
      -
    • DALLE-3 filters your content AFTER image creation
      • -
    • With prompt “dead cookies” you can reproduce inconsistent filtering over OpenAI API
      • -
    • 40% of cases with same “dead cookies” prompt stop through content filter and 60% reach us over API
      • -
    -
    - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • Evil LLMs available! Break GenAI Alignment through finetuning! - +

    Need for LLM Alignment transparency?

    Image 1

    TL;DR ⏱️

    @@ -958,12 +958,12 @@

    Tagged “blog”

  • Questions about alignment in LLMs
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource)
  • LLAMA2 13B is faster than LLAMA2 7B, according to NVIDIA benchmark! - +

    Interesting findings on NVIDIA's LLAMA 2 benchmark results.

    Image 1

    TL;DR ⏱️

    @@ -974,7 +974,7 @@

    Tagged “blog”

  • Seeking community insights
    • - (Read More)  (Original-Websource) + (Read More)  (Original-Websource) @@ -990,7 +990,7 @@

    Tagged “blog”

    © 2024 Carsten Felix Draschner, PhD

    - + diff --git a/tags/index.html b/tags/index.html index b9a56e4..dd4d42b 100644 --- a/tags/index.html +++ b/tags/index.html @@ -403,13 +403,13 @@

    Tags